Understanding API Uptime Monitoring#

API uptime refers to the percentage of time an API is available and functioning correctly. Monitoring involves repeatedly checking an API endpoint to determine its responsiveness and the nature of its response. Key metrics collected during monitoring include:

• Availability: Determined by the HTTP status code returned. A status code in the 2xx range (e.g., 200 OK) typically indicates success, while 4xx (client errors) or 5xx (server errors) indicate issues.
• Latency: The time taken for the API to respond to a request. High latency can indicate performance degradation even if the API is technically “available”.
• Response Content: Verification that the API returns the expected data or structure, providing a deeper check than just the status code.

Consistent monitoring generates data points over time, which can be analyzed and visualized to understand trends, identify recurring issues, and calculate uptime percentages.

Why Python and Streamlit for an Uptime Dashboard?#

Python is a versatile programming language with robust libraries for making HTTP requests (requests) and handling data (pandas). Streamlit is an open-source Python library that simplifies the creation of custom web applications and dashboards with minimal code, making it ideal for rapidly building interactive data visualizations without needing extensive web development knowledge.

Combining Python’s monitoring capabilities with Streamlit’s ease of dashboard creation offers an efficient approach to developing a custom API uptime monitoring tool tailored to specific needs.

Essential Concepts for the Dashboard#

Developing an API uptime dashboard involves several core components:

1. API Endpoint List: A defined list of URLs to be monitored.
2. Monitoring Logic: A script or function that sends requests to each API endpoint and records the result (status code, latency, timestamp).
3. Data Storage: A method to store the monitoring results persistently or in-memory for display.
4. Scheduling: A mechanism to run the monitoring logic at regular intervals.
5. Visualization: The dashboard interface that displays the collected data in a user-friendly format (tables, charts).

This guide focuses on the monitoring logic, data handling, and visualization using Python and Streamlit. External scheduling is a crucial operational consideration but sits outside the scope of the core Streamlit application code itself.

Building the Dashboard: A Step-by-Step Walkthrough#

This section details the process of creating the Python script and Streamlit application.

1
# Create a virtual environment
2
python -m venv venv
3
# Activate the environment (Linux/macOS)
4
source venv/bin/activate
5
# Activate the environment (Windows)
6
.\venv\Scripts\activate
7

8
# Install required libraries
9
pip install requests pandas streamlit

1
api_endpoints = [
2
    "https://www.example.com/api/status",
3
    "https://api.another-service.com/v1/health",
4
    "https://api.yet-another.org/check"
5
]

1
import requests
2
import time
3
from datetime import datetime
4

5
def check_api_status(url):
6
    """Checks the status and latency of a given URL."""
7
    status_code = None
8
    latency = None
9
    error = None
10
    timestamp = datetime.now().isoformat()
11

12
    try:
13
        start_time = time.time()
14
        response = requests.get(url, timeout=10) # Set a timeout
15
        end_time = time.time()
16

17
        status_code = response.status_code
18
        latency = (end_time - start_time) * 1000 # Latency in milliseconds
19

20
        # Basic check for successful status codes
21
        if not 200 <= status_code < 300:
22
            error = f"HTTP Status: {status_code}"
23

24
    except requests.exceptions.Timeout:
25
        status_code = "Timeout"
26
        error = "Request timed out"
27
    except requests.exceptions.ConnectionError:
28
        status_code = "Error"
29
        error = "Connection Error"
30
    except Exception as e:
31
        status_code = "Error"
32
        error = f"An unexpected error occurred: {e}"
33

34
    return {
35
        "timestamp": timestamp,
36
        "url": url,
37
        "status_code": status_code,
38
        "latency_ms": round(latency, 2) if latency is not None else None,
39
        "error": error,
40
        "is_available": (error is None and status_code != "Timeout" and status_code != "Error")
41
    }

1
import pandas as pd
2

3
# In-memory storage for demonstration
4
monitoring_results = []
5

6
# Function to perform checks for all APIs and store results
7
def perform_all_checks(endpoints):
8
    current_results = []
9
    for url in endpoints:
10
        result = check_api_status(url)
11
        current_results.append(result)
12
        monitoring_results.append(result) # Add to historical list
13
    return current_results # Return only the latest results for immediate display

1
import streamlit as st
2
import pandas as pd
3
import time # Import time for potential sleep
4
# Assume check_api_status and api_endpoints are defined above
5

6
st.set_page_config(layout="wide") # Use wide layout
7

8
st.title("API Uptime Dashboard")
9

10
# Function to display the latest status
11
def display_latest_status(latest_results):
12
    st.subheader("Latest Status Check")
13
    # Use st.status to show check progress (optional but good UI)
14
    with st.status("Checking APIs...", expanded=True) as status:
15
        results_list = []
16
        for url in api_endpoints:
17
            st.write(f"Checking {url}...")
18
            result = check_api_status(url)
19
            monitoring_results.append(result) # Add to the global history
20
            results_list.append(result) # Add to current batch
21
        status.update(label="Checks Complete!", state="complete", expanded=False)
22

23
    # Convert latest results to DataFrame for display
24
    if results_list:
25
        latest_df = pd.DataFrame(results_list)
26
        latest_df_display = latest_df[['url', 'status_code', 'latency_ms', 'error', 'is_available']]
27
        st.dataframe(latest_df_display, use_container_width=True)
28
    else:
29
         st.info("No latest check data available. Click 'Run Checks'.")
30

31

32
# Function to display historical data (from the global monitoring_results list)
33
def display_historical_data():
34
    st.subheader("Historical Data")
35
    if monitoring_results:
36
        history_df = pd.DataFrame(monitoring_results)
37
        st.dataframe(history_df, use_container_width=True)
38

39
        # Basic uptime percentage calculation (example)
40
        st.subheader("Uptime Summary")
41
        uptime_summary = history_df.groupby('url')['is_available'].value_counts(normalize=True).unstack(fill_value=0)
42
        uptime_summary['uptime_percentage'] = uptime_summary.get(True, 0) * 100 # Use .get(True, 0) in case no 'True'
43
        st.dataframe(uptime_summary[['uptime_percentage']].round(2), use_container_width=True)
44

45
        # Example plot: Latency over time for one API (select first one for simplicity)
46
        if not history_df.empty and api_endpoints:
47
             st.subheader(f"Latency Trend for {api_endpoints[0]}")
48
             # Filter for the first API and successful checks with latency data
49
             first_api_history = history_df[(history_df['url'] == api_endpoints[0]) & (history_df['latency_ms'].notna()) & (history_df['is_available'] == True)].copy()
50
             # Convert timestamp to datetime for plotting
51
             first_api_history['timestamp'] = pd.to_datetime(first_api_history['timestamp'])
52
             if not first_api_history.empty:
53
                 st.line_chart(first_api_history.set_index('timestamp')['latency_ms'])
54
             else:
55
                 st.info(f"No successful checks with latency data for {api_endpoints[0]} to plot.")
56

57

58
    else:
59
        st.info("No historical data yet. Run some checks first.")
60

61
# --- Streamlit App Structure ---
62

63
# Global variable to store results in memory (resets on app rerun)
64
if 'monitoring_results' not in st.session_state:
65
    st.session_state.monitoring_results = []
66
monitoring_results = st.session_state.monitoring_results # Link global variable to session state
67

68
# Add a button to manually trigger checks
69
if st.button("Run Checks Now"):
70
     st.session_state.latest_check_results = [] # Store latest check results separately for immediate display
71
     with st.status("Running checks...", expanded=True) as status:
72
        for url in api_endpoints:
73
            st.write(f"Checking {url}...")
74
            result = check_api_status(url)
75
            st.session_state.monitoring_results.append(result) # Add to historical list
76
            st.session_state.latest_check_results.append(result) # Add to the latest results list
77
        status.update(label="Checks Complete!", state="complete", expanded=False)
78

79
     # Display latest results after the button click
80
     if st.session_state.latest_check_results:
81
        latest_df = pd.DataFrame(st.session_state.latest_check_results)
82
        latest_df_display = latest_df[['url', 'status_code', 'latency_ms', 'error', 'is_available']]
83
        st.subheader("Latest Check Results")
84
        st.dataframe(latest_df_display, use_container_width=True)
85

86

87
# Display historical data and summary below the button/latest results
88
display_historical_data()
89

90
# Note: For automatic, periodic checks, this Streamlit app is not the scheduler.
91
# A separate process/script would need to run the checks and potentially
92
# update a persistent data store (file, DB) that this app reads.

1
streamlit run dashboard.py

Data Storage and Visualization Options#

Data Storage:

• In-Memory (st.session_state): Simplest for demonstration. Data is lost when the Streamlit app restarts.
• File Storage (CSV/JSON): Data persists between runs. Requires reading the file on startup and writing after each check. Scalability limited by file size and complexity of concurrent access if used with an external scheduler.
• Simple Database (SQLite): More robust persistence. Supports concurrent access better than files. Requires a library like sqlite3. Ideal when an external script performs checks and writes to the DB, and the Streamlit app reads from it.

Visualization:

• st.dataframe(): Excellent for displaying raw tabular data (latest checks, historical log).
• st.line_chart() / st.bar_chart(): Useful for visualizing trends like latency over time or the distribution of status codes.
• Conditional Formatting: Pandas DataFrames displayed by st.dataframe can be styled to highlight status codes or errors using DataFrame styling methods before passing to Streamlit.
• Custom Components: Streamlit allows embedding HTML or using custom components for more advanced visualizations if needed.

Real-World Application and Considerations#

Consider a software company providing a service that integrates with multiple third-party APIs (e.g., payment gateways, shipping carriers, social media platforms). The reliability of these external APIs directly impacts the company’s service quality.

• Use Case: Building a simple API uptime dashboard allows the operations or development team to have immediate visibility into the status of these critical dependencies. Instead of waiting for customer reports or internal errors, they can proactively identify outages or performance issues with a quick glance at the dashboard.
• Implementation:
  1. A separate Python script runs periodically (e.g., every 5 minutes via cron or a cloud scheduler like AWS EventBridge).
  2. This script uses the check_api_status function for each required endpoint.
  3. Results are appended to a persistent data store, such as a SQLite database file or written to a simple log file (e.g., in JSON format, one object per line).
  4. The Streamlit application dashboard.py reads the latest data from this persistent store on startup and provides a button to refresh the view, querying the store again. It displays the current status, historical logs, and potentially calculates uptime percentages and plots latency trends from the stored data.

This decoupled architecture (scheduler + monitoring script separate from the dashboard) is more scalable and reliable for continuous monitoring than relying solely on a Streamlit app’s interactivity.

Further Considerations:

• Authentication: Monitoring private APIs requires handling authentication (API keys, OAuth tokens). The requests library supports various authentication methods.
• Advanced Checks: Beyond basic HTTP GET, monitoring might involve checking specific response headers, validating JSON structure, or performing POST requests.
• Alerting: A dashboard provides visibility, but for critical APIs, integrating alerting (email, Slack, PagerDuty) based on failed checks is essential. This requires additional logic in the monitoring script.
• Scaling: Monitoring hundreds or thousands of APIs requires a more robust architecture, potentially involving message queues, distributed workers, and a more powerful database, moving beyond the scope of a simple Python/Streamlit app towards dedicated monitoring solutions.

Key Takeaways#

• Building an API uptime dashboard provides crucial visibility into the health of critical service dependencies.
• Python, with libraries like requests and pandas, is effective for implementing monitoring logic and data handling.
• Streamlit enables rapid development of interactive web dashboards with minimal code, ideal for visualizing API status.
• A basic dashboard can be built using Streamlit’s data display (st.dataframe) and charting (st.line_chart) capabilities.
• For continuous monitoring, the check scheduling should ideally be handled by a separate process or service, not the Streamlit application itself.
• Persistent data storage (files, SQLite) is necessary for retaining historical monitoring data across application runs.
• Real-world applications require considering authentication, more complex checks, alerting, and potential scaling requirements.