AI Agents Hands-On Demo: Predicting 30-Day Hospital Readmissions for Diabetic Patients

Dataset: Diabetes 130-US Hospitals for Years 1999–2008 (UCI Machine Learning Repository), 101,766 hospital encounters.

• Under Medicare’s Hospital Readmissions Reduction Program (HRRP), hospitals are financially penalized when too many patients are readmitted within 30 days of discharge. Spotting high-risk patients before discharge lets care teams step in with follow-up calls, medication reviews or home-health referrals.

Contents

Part	What we do	Why
1. Data processing	Load and clean the data	Real clinical data is messy
2. Traditional ML	Train logistic regression and gradient boosting	A model gives a score, and a human does all the analysis
3. Build an AI agent	Give an LLM (Claude) a set of analysis tools and a loop	The agent decides which analyses to run
4. Agent demos	Ask the agent questions in plain English	Multi-step reasoning, explanation, and actionable output
5. Comparison & limitations	Traditional ML vs. agents	When to use each, and the risks

What is an AI agent?

An AI agent is an LLM that works in a loop: it can take actions (call tools), observe the results, and decide what to do next until it reaches its goal. This is the ReAct pattern (Reason + Act).

The four building blocks (we’ll see each one in code): * 1. Model (LLM): the reasoning engine. We use Anthropic’s Claude via the API. * 2. Tools: functions the agent may call, each described to the model with a name, description and input schema. * 3. Memory / context: the running message history, so each step builds on the earlier ones. * 4. Orchestration loop: our code that sends messages, runs requested tools, returns the results, and enforces guardrails (such as a maximum number of steps).

0. Setup

import os, json, warnings
from pathlib import Path

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import HistGradientBoostingClassifier

from sklearn.metrics import (
    accuracy_score,
    balanced_accuracy_score,
    confusion_matrix,
    classification_report,
    precision_score,
    recall_score,
    f1_score,
    roc_curve,
    roc_auc_score
)

import anthropic

warnings.filterwarnings("ignore")
pd.set_option("display.max_columns", 60)
plt.rcParams["figure.dpi"] = 110

# Random seed for reproducibility
RANDOM_STATE = 25
np.random.seed(RANDOM_STATE)

DATA_PATH = Path("data/diabetic_data.csv")
OUTPUT_DIR = Path("outputs"); OUTPUT_DIR.mkdir(exist_ok=True)

1. Data Processing

Each row is one hospital stay (“encounter”) of a diabetic patient. The target column readmitted has three values: <30 (readmitted within 30 days), >30 (readmitted later), and NO. The HRRP penalty is for 30-day readmissions, so we predict <30 vs. everything else.

# '?' marks missing values. keep_default_na=False stops pandas from turning the
# literal string "None" (= "test not performed") into a missing value.

raw = pd.read_csv(DATA_PATH, na_values="?", keep_default_na=False, low_memory=False)
raw.head()

	encounter_id	patient_nbr	race	gender	age	weight	admission_type_id	discharge_disposition_id	admission_source_id	time_in_hospital	payer_code	medical_specialty	num_lab_procedures	num_procedures	num_medications	number_outpatient	number_emergency	number_inpatient	diag_1	diag_2	diag_3	number_diagnoses	max_glu_serum	A1Cresult	metformin	repaglinide	nateglinide	chlorpropamide	glimepiride	acetohexamide	glipizide	glyburide	tolbutamide	pioglitazone	rosiglitazone	acarbose	miglitol	troglitazone	tolazamide	examide	citoglipton	insulin	glyburide-metformin	glipizide-metformin	glimepiride-pioglitazone	metformin-rosiglitazone	metformin-pioglitazone	change	diabetesMed	readmitted
0	2278392	8222157	Caucasian	Female	[0-10)	NaN	6	25	1	1	NaN	Pediatrics-Endocrinology	41	0	1	0	0	0	250.83	NaN	NaN	1	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	NO
1	149190	55629189	Caucasian	Female	[10-20)	NaN	1	1	7	3	NaN	NaN	59	0	18	0	0	0	276	250.01	255	9	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Up	No	No	No	No	No	Ch	Yes	>30
2	64410	86047875	AfricanAmerican	Female	[20-30)	NaN	1	1	7	2	NaN	NaN	11	5	13	2	0	1	648	250	V27	6	None	None	No	No	No	No	No	No	Steady	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Yes	NO
3	500364	82442376	Caucasian	Male	[30-40)	NaN	1	1	7	2	NaN	NaN	44	1	16	0	0	0	8	250.43	403	7	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Up	No	No	No	No	No	Ch	Yes	NO
4	16680	42519267	Caucasian	Male	[40-50)	NaN	1	1	7	1	NaN	NaN	51	0	8	0	0	0	197	157	250	5	None	None	No	No	No	No	No	No	Steady	No	No	No	No	No	No	No	No	No	No	Steady	No	No	No	No	No	Ch	Yes	NO

print(f"Raw data: {raw.shape[0]:,} encounters x {raw.shape[1]} columns")
print("\nTarget distribution:")
print(raw["readmitted"].value_counts(normalize=True).mul(100).round(1).astype(str) + "%")
print("\nColumns with missing values (% missing):")
print((raw.isna().mean() * 100).round(1)[lambda s: s > 0].sort_values(ascending=False))

Raw data: 101,766 encounters x 50 columns

Target distribution:
readmitted
NO     53.9%
>30    34.9%
<30    11.2%
Name: proportion, dtype: object

Columns with missing values (% missing):
weight               96.9
medical_specialty    49.1
payer_code           39.6
race                  2.2
diag_3                1.4
diag_2                0.4
dtype: float64

Cleaning decisions (following Strack et al., 2014, the paper that introduced this dataset): 1. Remove patients who died or went to hospice. They cannot be readmitted, so they would bias the model. 2. Keep only each patient’s first encounter. Many patients appear several times, and keeping repeats would leak information between training and test sets. 3. Drop weight (97% missing) and payer_code (40% missing, not clinically meaningful). 4. Group high-cardinality codes: ICD-9 diagnosis codes become disease categories, and admission/discharge IDs become readable groups.

df = raw.copy()

# 1) remove encounters ending in death or hospice (discharge_disposition_id 11, 13, 14, 19, 20, 21)
df = df[~df["discharge_disposition_id"].isin([11, 13, 14, 19, 20, 21])]

# 2) keep first encounter per patient
df = df.sort_values("encounter_id").drop_duplicates("patient_nbr", keep="first")

# remove 3 rows with invalid gender
df = df[df["gender"] != "Unknown/Invalid"]

# 3) Target: 1 = readmitted within 30 days
df["readmit_30"] = (df["readmitted"] == "<30").astype(int)

# 4) Feature engineering 

# map an ICD-9 code to a broad disease category. There are ~700 codes the # is part of a range that determines the greater issue, so 
# like 390-459 is cardiac issues
def icd9_group(code):
    if pd.isna(code):
        return "Missing"
    if code.startswith(("V", "E")):
        return "Other"
    c = float(code)
    if 390 <= c <= 459 or int(c) == 785: return "Circulatory"
    if 460 <= c <= 519 or int(c) == 786: return "Respiratory"
    if 520 <= c <= 579 or int(c) == 787: return "Digestive"
    if int(c) == 250:                    return "Diabetes"
    if 800 <= c <= 999:                  return "Injury"
    if 710 <= c <= 739:                  return "Musculoskeletal"
    if 580 <= c <= 629 or int(c) == 788: return "Genitourinary"
    if 140 <= c <= 239:                  return "Neoplasms"
    return "Other"

# turns id codes (admission type, admission course... into a few readable groups
# like "home", "skilled nursing / longterm care"
# about ~ 30 discharge codes, monay rare or duplicates. readable names make it easier to understand agents
def group_ids(series, mapping, default="Other/Unknown"): 
    out = pd.Series(default, index=series.index)
    for label, ids in mapping.items():
        out[series.isin(ids)] = label
    return out

# list of 23 diabetes medicationn columns
# used to build `num_diabetes_meds` 
MED_COLS = ["metformin", "repaglinide", "nateglinide", "chlorpropamide", "glimepiride", "acetohexamide",
            "glipizide", "glyburide", "tolbutamide", "pioglitazone", "rosiglitazone", "acarbose", "miglitol",
            "troglitazone", "tolazamide", "examide", "citoglipton", "insulin", "glyburide-metformin",
            "glipizide-metformin", "glimepiride-pioglitazone", "metformin-rosiglitazone", "metformin-pioglitazone"]

# building patients table
patients = pd.DataFrame({
    "encounter_id":       df["encounter_id"],
    "age_years":          df["age"].str.extract(r"\[(\d+)-")[0].astype(int) + 5,   # "[70-80)" -> 75 trasform from text ranges
    "gender":             df["gender"],
    "race":               df["race"].fillna("Unknown"), # missing becomes unknown
   
    # group_ids ^ 
    "admission_type":     group_ids(df["admission_type_id"], {"Emergency": [1], "Urgent": [2], "Elective": [3]}), 
    "admission_source":   group_ids(df["admission_source_id"], {"Emergency Room": [7], "Referral": [1, 2, 3], 
                                                                "Transfer": [4, 5, 6, 10, 18, 22, 25, 26]}), 
    "discharge_to":       group_ids(df["discharge_disposition_id"], {
                              "Home": [1], "Home with health services": [6, 8],
                              "Skilled nursing / long-term care": [3, 4, 15, 23, 24],
                              "Other inpatient / rehab facility": [2, 5, 9, 10, 22, 27, 28, 29, 30],
                              "Left against medical advice": [7]}),
    "time_in_hospital":   df["time_in_hospital"],
    "num_lab_procedures": df["num_lab_procedures"],
    "num_procedures":     df["num_procedures"],
    "num_medications":    df["num_medications"],
    "number_outpatient":  df["number_outpatient"],
    "number_emergency":   df["number_emergency"],
    "number_inpatient":   df["number_inpatient"],
    "number_diagnoses":   df["number_diagnoses"],
    
    # `icd9_group` applied to diag_1
    "primary_diagnosis":  df["diag_1"].apply(icd9_group),
    
    "medical_specialty":  df["medical_specialty"].fillna("Unknown"), # missing becomes unknown
    
    # "none" means not tested, changed for clarification
    "A1C_result":         df["A1Cresult"].replace("None", "Not tested"), 
    "max_glucose":        df["max_glu_serum"].replace("None", "Not tested"),
    
    "insulin":            df["insulin"],
    "num_diabetes_meds":  (df[MED_COLS] != "No").sum(axis=1),
    
    # changed for readability
    "med_change":         df["change"].map({"Ch": "Yes", "No": "No"}), 
    
    "on_diabetes_med":    df["diabetesMed"],
    "readmit_30":         df["readmit_30"],
}).reset_index(drop=True)

# keep the 8 most common specialties, lump the rest
top_specs = patients["medical_specialty"].value_counts().index[:8]
patients.loc[~patients["medical_specialty"].isin(top_specs), "medical_specialty"] = "Other"

NUMERIC = ["age_years", "time_in_hospital", "num_lab_procedures", "num_procedures", "num_medications",
           "number_outpatient", "number_emergency", "number_inpatient", "number_diagnoses", "num_diabetes_meds"]
CATEGORICAL = ["gender", "race", "admission_type", "admission_source", "discharge_to", "primary_diagnosis",
               "medical_specialty", "A1C_result", "max_glucose", "insulin", "med_change", "on_diabetes_med"]
FEATURES = NUMERIC + CATEGORICAL

# FINALLY!!!:
print(f"After cleaning: {len(patients):,} unique patients, {len(FEATURES)} features")
print(f"30-day readmission rate: {patients['readmit_30'].mean():.1%}")
patients.head()

After cleaning: 69,987 unique patients, 22 features


30-day readmission rate: 9.0%

	encounter_id	age_years	gender	race	admission_type	admission_source	discharge_to	time_in_hospital	num_lab_procedures	num_procedures	num_medications	number_outpatient	number_emergency	number_inpatient	number_diagnoses	primary_diagnosis	medical_specialty	A1C_result	max_glucose	insulin	num_diabetes_meds	med_change	on_diabetes_med	readmit_30
0	12522	85	Female	Caucasian	Urgent	Transfer	Home	13	68	2	28	0	0	0	8	Circulatory	Unknown	Not tested	Not tested	Steady	2	Yes	Yes	0
1	15738	95	Female	Caucasian	Elective	Transfer	Skilled nursing / long-term care	12	33	3	18	0	0	0	8	Circulatory	InternalMedicine	Not tested	Not tested	Steady	2	Yes	Yes	0
2	16680	45	Male	Caucasian	Emergency	Emergency Room	Home	1	51	0	8	0	0	0	5	Neoplasms	Unknown	Not tested	Not tested	Steady	2	Yes	Yes	0
3	28236	45	Female	AfricanAmerican	Emergency	Emergency Room	Home	9	47	2	17	0	0	0	9	Diabetes	Unknown	Not tested	Not tested	Steady	1	No	Yes	0
4	35754	55	Male	Caucasian	Urgent	Referral	Home	3	31	6	16	0	0	0	9	Circulatory	Unknown	Not tested	Not tested	Steady	1	No	Yes	0

2. Traditional machine learning baseline

This is the classic approach: turn the features into numbers, split into train/test sets, fit a model, and measure it. We use: - Logistic regression: simple and interpretable (each feature gets a weight). - Gradient boosting: a stronger tree-based model.

Metrics. AUC (area under the ROC curve; 0.5 = random guessing, 1.0 = perfect). * Also report a practical metric: if the care team can only follow up with the 20% of patients that the model ranks as the most at risk, what share of all 30-day readmissions do they reach?

# separate predictors and target
# target is "readmit_30" (1 = readmitted within 30 days)
# the 22 engineered features are the predictors
X = patients[FEATURES]
y = patients["readmit_30"]

print("Predictor shape:", X.shape)
print("Target shape:", y.shape)

Predictor shape: (69987, 22)
Target shape: (69987,)

# numerical and categorical variables
print("Numerical variables:")
print(NUMERIC)

print("Categorical variables:")
print(CATEGORICAL)

Numerical variables:
['age_years', 'time_in_hospital', 'num_lab_procedures', 'num_procedures', 'num_medications', 'number_outpatient', 'number_emergency', 'number_inpatient', 'number_diagnoses', 'num_diabetes_meds']
Categorical variables:
['gender', 'race', 'admission_type', 'admission_source', 'discharge_to', 'primary_diagnosis', 'medical_specialty', 'A1C_result', 'max_glucose', 'insulin', 'med_change', 'on_diabetes_med']

# examine target variable
print(y.value_counts())

print("Proportions:")
print(y.value_counts(normalize=True))

readmit_30
0    63702
1     6285
Name: count, dtype: int64
Proportions:
readmit_30
0    0.910198
1    0.089802
Name: proportion, dtype: float64

# one-hot encode the categorical variables
X_encoded = pd.get_dummies(X, columns=CATEGORICAL, dtype=float)

print("Variables before encoding:", X.shape[1])
print("Variables after encoding:", X_encoded.shape[1])

Variables before encoding: 22
Variables after encoding: 67

# train test split
X_train, X_test, y_train, y_test = train_test_split(
    X_encoded,
    y,
    test_size=0.20,
    random_state=RANDOM_STATE,
    stratify=y
)

# row positions of the train/test patients (used by the agent tools later)
idx_train = X_train.index.to_numpy()
idx_test = X_test.index.to_numpy()

print("Training observations:", X_train.shape[0])
print("Testing observations:", X_test.shape[0])

print("Training class proportions:")
print(y_train.value_counts(normalize=True))

print("Testing class proportions:")
print(y_test.value_counts(normalize=True))

Training observations: 55989
Testing observations: 13998
Training class proportions:
readmit_30
0    0.910197
1    0.089803
Name: proportion, dtype: float64
Testing class proportions:
readmit_30
0    0.910201
1    0.089799
Name: proportion, dtype: float64

# scale numerical variables
# scaler is fit on the training data only - prevents data leakage
scaler = StandardScaler().fit(X_train[NUMERIC])

X_train = X_train.copy()
X_test = X_test.copy()
X_train[NUMERIC] = scaler.transform(X_train[NUMERIC])
X_test[NUMERIC] = scaler.transform(X_test[NUMERIC])

X_train[NUMERIC].describe().round(2)

	age_years	time_in_hospital	num_lab_procedures	num_procedures	num_medications	number_outpatient	number_emergency	number_inpatient	number_diagnoses	num_diabetes_meds
count	55989.00	55989.00	55989.00	55989.00	55989.00	55989.00	55989.00	55989.00	55989.00	55989.00
mean	0.00	0.00	0.00	0.00	0.00	-0.00	0.00	0.00	0.00	0.00
std	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00
min	-3.78	-1.12	-2.10	-0.81	-1.77	-0.26	-0.20	-0.29	-3.12	-1.26
25%	-0.65	-0.78	-0.59	-0.81	-0.68	-0.26	-0.20	-0.29	-0.61	-0.20
50%	-0.03	-0.43	0.06	-0.24	-0.20	-0.26	-0.20	-0.29	0.39	-0.20
75%	0.60	0.59	0.71	0.32	0.52	-0.26	-0.20	-0.29	0.89	0.86
max	1.85	3.31	4.47	2.60	7.89	38.59	78.39	19.92	4.39	5.10

Confusion Matrix & Accuracy Metrics

AUC measures how well the models “rank” patients. To get a confusion matric, we need a yes/no prediction, so we flag the riskiest 20% of patients as “high risk” (the same group the care team could follow up with)

# logistic regression
# class_weight="balanced" because readmissions are rare (9%)
logistic_regression = LogisticRegression(
    max_iter=2000,
    C=0.5,
    class_weight="balanced"
)

logistic_regression.fit(X_train, y_train)

LogisticRegression(C=0.5, class_weight='balanced', max_iter=2000)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.