AI Agents Hands-On Demo: Predicting 30-Day Hospital Readmissions for Diabetic Patients

Dataset: Diabetes 130-US Hospitals for Years 1999–2008 (UCI Machine Learning Repository), 101,766 hospital encounters.

Under Medicare’s Hospital Readmissions Reduction Program (HRRP), hospitals are financially penalized when too many patients are readmitted within 30 days of discharge. Spotting high-risk patients before discharge lets care teams step in with follow-up calls, medication reviews or home-health referrals.

Part	What we do	Why
1. Data processing	Load and clean the data	Real clinical data is messy
2. Traditional ML	Train logistic regression and gradient boosting	A model gives a score, and a human does all the analysis
3. Build an AI agent	Give an LLM (Claude) a set of analysis tools and a loop	The agent decides which analyses to run
4. Agent demos	Ask the agent questions in plain English	Multi-step reasoning, explanation, and actionable output
5. Comparison & limitations	Traditional ML vs. agents	When to use each, and the risks

What is an AI agent?

An AI agent is an LLM that works in a loop: it can take actions (call tools), observe the results, and decide what to do next until it reaches its goal. This is the ReAct pattern (Reason + Act).

The four building blocks (we’ll see each one in code): 1. Model (LLM): the reasoning engine. We use Anthropic’s Claude via the API. 2. Tools: functions the agent may call, each described to the model with a name, description and input schema. 3. Memory / context: the running message history, so each step builds on the earlier ones. 4. Orchestration loop: our code that sends messages, runs requested tools, returns the results, and enforces guardrails (such as a maximum number of steps).

0. Setup

import os, json, warnings
from pathlib import Path

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.metrics import roc_auc_score, roc_curve
import anthropic

warnings.filterwarnings("ignore")
pd.set_option("display.max_columns", 60)
plt.rcParams["figure.dpi"] = 110
RANDOM_STATE = 25
DATA_PATH = Path("data/diabetic_data.csv")
OUTPUT_DIR = Path("outputs"); OUTPUT_DIR.mkdir(exist_ok=True)

1. Data Processing

Each row is one hospital stay (“encounter”) of a diabetic patient. The target column readmitted has three values: <30 (readmitted within 30 days), >30 (readmitted later), and NO. The HRRP penalty is for 30-day readmissions, so we predict <30 vs. everything else.

# '?' marks missing values. keep_default_na=False stops pandas from turning the
# literal string "None" (= "test not performed") into a missing value.

raw = pd.read_csv(DATA_PATH, na_values="?", keep_default_na=False, low_memory=False)
raw.head()

	encounter_id	patient_nbr	race	gender	age	weight	admission_type_id	discharge_disposition_id	admission_source_id	time_in_hospital	payer_code	medical_specialty	num_lab_procedures	num_procedures	num_medications	number_outpatient	number_inpatient	diag_1	diag_2	diag_3	number_diagnoses	max_glu_serum	A1Cresult	metformin	repaglinide	nateglinide	chlorpropamide	glimepiride	acetohexamide	glipizide	glyburide	tolbutamide	pioglitazone	rosiglitazone	acarbose	miglitol	troglitazone	tolazamide	examide	citoglipton	insulin	glyburide-metformin	glipizide-metformin	glimepiride-pioglitazone	metformin-rosiglitazone	metformin-pioglitazone	change	diabetesMed	readmitted
0	2278392	8222157	Caucasian	Female	[0-10)	NaN	6	25	1	1	NaN	Pediatrics-Endocrinology	41	0	1	0	0	250.83	NaN	NaN	1	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	NO
1	149190	55629189	Caucasian	Female	[10-20)	NaN	1	1	7	3	NaN	NaN	59	0	18	0	0	276	250.01	255	9	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Up	No	No	No	No	No	Ch	Yes	>30
2	64410	86047875	AfricanAmerican	Female	[20-30)	NaN	1	1	7	2	NaN	NaN	11	5	13	2	1	648	250	V27	6	None	None	No	No	No	No	No	No	Steady	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Yes	NO
3	500364	82442376	Caucasian	Male	[30-40)	NaN	1	1	7	2	NaN	NaN	44	1	16	0	0	8	250.43	403	7	None	None	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	No	Up	No	No	No	No	No	Ch	Yes	NO
4	16680	42519267	Caucasian	Male	[40-50)	NaN	1	1	7	1	NaN	NaN	51	0	8	0	0	197	157	250	5	None	None	No	No	No	No	No	No	Steady	No	No	No	No	No	No	No	No	No	No	Steady	No	No	No	No	No	Ch	Yes	NO

print(f"Raw data: {raw.shape[0]:,} encounters x {raw.shape[1]} columns")
print("\nTarget distribution:")
print(raw["readmitted"].value_counts(normalize=True).mul(100).round(1).astype(str) + "%")
print("\nColumns with missing values (% missing):")
print((raw.isna().mean() * 100).round(1)[lambda s: s > 0].sort_values(ascending=False))

Raw data: 101,766 encounters x 50 columns

Target distribution:
readmitted
NO     53.9%
>30    34.9%
<30    11.2%
Name: proportion, dtype: object

Columns with missing values (% missing):
weight               96.9
medical_specialty    49.1
payer_code           39.6
race                  2.2
diag_3                1.4
diag_2                0.4
dtype: float64

Cleaning decisions (following Strack et al., 2014, the paper that introduced this dataset): 1. Remove patients who died or went to hospice. They cannot be readmitted, so they would bias the model. 2. Keep only each patient’s first encounter. Many patients appear several times, and keeping repeats would leak information between training and test sets. 3. Drop weight (97% missing) and payer_code (40% missing, not clinically meaningful). 4. Group high-cardinality codes: ICD-9 diagnosis codes become disease categories, and admission/discharge IDs become readable groups.

df = raw.copy()

# 1) remove encounters ending in death or hospice (discharge_disposition_id 11, 13, 14, 19, 20, 21)
df = df[~df["discharge_disposition_id"].isin([11, 13, 14, 19, 20, 21])]

# 2) keep first encounter per patient
df = df.sort_values("encounter_id").drop_duplicates("patient_nbr", keep="first")

# remove 3 rows with invalid gender
df = df[df["gender"] != "Unknown/Invalid"]

# 3) Target: 1 = readmitted within 30 days
df["readmit_30"] = (df["readmitted"] == "<30").astype(int)

# 4) Feature engineering 

# map an ICD-9 code to a broad disease category. There are ~700 codes the # is part of a range that determines the greater issue, so 
# like 390-459 is cardiac issues
def icd9_group(code):
    if pd.isna(code):
        return "Missing"
    if code.startswith(("V", "E")):
        return "Other"
    c = float(code)
    if 390 <= c <= 459 or int(c) == 785: return "Circulatory"
    if 460 <= c <= 519 or int(c) == 786: return "Respiratory"
    if 520 <= c <= 579 or int(c) == 787: return "Digestive"
    if int(c) == 250:                    return "Diabetes"
    if 800 <= c <= 999:                  return "Injury"
    if 710 <= c <= 739:                  return "Musculoskeletal"
    if 580 <= c <= 629 or int(c) == 788: return "Genitourinary"
    if 140 <= c <= 239:                  return "Neoplasms"
    return "Other"

# turns id codes (admission type, admission course... into a few readable groups
# like "home", "skilled nursing / longterm care"
# about ~ 30 discharge codes, monay rare or duplicates. readable names make it easier to understand agents
def group_ids(series, mapping, default="Other/Unknown"): 
    out = pd.Series(default, index=series.index)
    for label, ids in mapping.items():
        out[series.isin(ids)] = label
    return out

# list of 23 diabetes medicationn columns
# used to build `num_diabetes_meds` 
MED_COLS = ["metformin", "repaglinide", "nateglinide", "chlorpropamide", "glimepiride", "acetohexamide",
            "glipizide", "glyburide", "tolbutamide", "pioglitazone", "rosiglitazone", "acarbose", "miglitol",
            "troglitazone", "tolazamide", "examide", "citoglipton", "insulin", "glyburide-metformin",
            "glipizide-metformin", "glimepiride-pioglitazone", "metformin-rosiglitazone", "metformin-pioglitazone"]

# building patients table
patients = pd.DataFrame({
    "encounter_id":       df["encounter_id"],
    "age_years":          df["age"].str.extract(r"\[(\d+)-")[0].astype(int) + 5,   # "[70-80)" -> 75 trasform from text ranges
    "gender":             df["gender"],
    "race":               df["race"].fillna("Unknown"), # missing becomes unknown
   
    # group_ids ^ 
    "admission_type":     group_ids(df["admission_type_id"], {"Emergency": [1], "Urgent": [2], "Elective": [3]}), 
    "admission_source":   group_ids(df["admission_source_id"], {"Emergency Room": [7], "Referral": [1, 2, 3], 
                                                                "Transfer": [4, 5, 6, 10, 18, 22, 25, 26]}), 
    "discharge_to":       group_ids(df["discharge_disposition_id"], {
                              "Home": [1], "Home with health services": [6, 8],
                              "Skilled nursing / long-term care": [3, 4, 15, 23, 24],
                              "Other inpatient / rehab facility": [2, 5, 9, 10, 22, 27, 28, 29, 30],
                              "Left against medical advice": [7]}),
    "time_in_hospital":   df["time_in_hospital"],
    "num_lab_procedures": df["num_lab_procedures"],
    "num_procedures":     df["num_procedures"],
    "num_medications":    df["num_medications"],
    "number_outpatient":  df["number_outpatient"],
    "number_emergency":   df["number_emergency"],
    "number_inpatient":   df["number_inpatient"],
    "number_diagnoses":   df["number_diagnoses"],
    
    # `icd9_group` applied to diag_1
    "primary_diagnosis":  df["diag_1"].apply(icd9_group),
    
    "medical_specialty":  df["medical_specialty"].fillna("Unknown"), # missing becomes unknown
    
    # "none" means not tested, changed for clarification
    "A1C_result":         df["A1Cresult"].replace("None", "Not tested"), 
    "max_glucose":        df["max_glu_serum"].replace("None", "Not tested"),
    
    "insulin":            df["insulin"],
    "num_diabetes_meds":  (df[MED_COLS] != "No").sum(axis=1),
    
    # changed for readability
    "med_change":         df["change"].map({"Ch": "Yes", "No": "No"}), 
    
    "on_diabetes_med":    df["diabetesMed"],
    "readmit_30":         df["readmit_30"],
}).reset_index(drop=True)

# keep the 8 most common specialties, lump the rest
top_specs = patients["medical_specialty"].value_counts().index[:8]
patients.loc[~patients["medical_specialty"].isin(top_specs), "medical_specialty"] = "Other"

NUMERIC = ["age_years", "time_in_hospital", "num_lab_procedures", "num_procedures", "num_medications",
           "number_outpatient", "number_emergency", "number_inpatient", "number_diagnoses", "num_diabetes_meds"]
CATEGORICAL = ["gender", "race", "admission_type", "admission_source", "discharge_to", "primary_diagnosis",
               "medical_specialty", "A1C_result", "max_glucose", "insulin", "med_change", "on_diabetes_med"]
FEATURES = NUMERIC + CATEGORICAL

# FINALLY!!!:
print(f"After cleaning: {len(patients):,} unique patients, {len(FEATURES)} features")
print(f"30-day readmission rate: {patients['readmit_30'].mean():.1%}")
patients.head()

After cleaning: 69,987 unique patients, 22 features
30-day readmission rate: 9.0%

	encounter_id	age_years	gender	race	admission_type	admission_source	discharge_to	time_in_hospital	num_lab_procedures	num_procedures	num_medications	number_diagnoses	primary_diagnosis	medical_specialty	A1C_result	max_glucose	insulin	num_diabetes_meds	med_change	on_diabetes_med
0	12522	85	Female	Caucasian	Urgent	Transfer	Home	13	68	2	28	8	Circulatory	Unknown	Not tested	Not tested	Steady	2	Yes	Yes
1	15738	95	Female	Caucasian	Elective	Transfer	Skilled nursing / long-term care	12	33	3	18	8	Circulatory	InternalMedicine	Not tested	Not tested	Steady	2	Yes	Yes
2	16680	45	Male	Caucasian	Emergency	Emergency Room	Home	1	51	0	8	5	Neoplasms	Unknown	Not tested	Not tested	Steady	2	Yes	Yes
3	28236	45	Female	AfricanAmerican	Emergency	Emergency Room	Home	9	47	2	17	9	Diabetes	Unknown	Not tested	Not tested	Steady	1	No	Yes
4	35754	55	Male	Caucasian	Urgent	Referral	Home	3	31	6	16	9	Circulatory	Unknown	Not tested	Not tested	Steady	1	No	Yes

2. Traditional machine learning baseline

This is the classic approach: turn the features into numbers, split into train/test sets, fit a model, and measure it. We use: - Logistic regression: simple and interpretable (each feature gets a weight). - Gradient boosting: a stronger tree-based model.

Metrics. AUC (area under the ROC curve; 0.5 = random guessing, 1.0 = perfect). * Also report a practical metric: if the care team can only follow up with the 20% of patients that the model ranks as the most at risk, what share of all 30-day readmissions do they reach?

# turn categories into 0/1 columns (one-hot encoding) so the models can use them
X_all = pd.get_dummies(patients[FEATURES], columns=CATEGORICAL, dtype=float)
y_all = patients["readmit_30"].values

# 80% train / 20% test. stratify keeps the same ~9% readmission rate in both
idx_train, idx_test = train_test_split(np.arange(len(patients)), test_size=0.2,
                    stratify=y_all, random_state=RANDOM_STATE)

# scale the numeric columns so they're on a similar range
# fit the scaler on train only, so no info from the test set leaks in
scaler = StandardScaler().fit(X_all.iloc[idx_train][NUMERIC])
X_scaled = X_all.copy()
X_scaled[NUMERIC] = scaler.transform(X_all[NUMERIC])

X_train, X_test = X_scaled.iloc[idx_train], X_scaled.iloc[idx_test]
y_train, y_test = y_all[idx_train], y_all[idx_test]

# the two models
# class_weight="balanced" -> readmissions are rare (9%), so give them more weight
models = {
    "logistic_regression": LogisticRegression(max_iter=2000, C=0.5, class_weight="balanced"),
    "gradient_boosting":   HistGradientBoostingClassifier(max_iter=300, learning_rate=0.05,
                            max_leaf_nodes=31, random_state=RANDOM_STATE),
}

# train each model, score the test set, and save the metrics
test_scores, metrics = {}, {}

for name, model in models.items():
    model.fit(X_train, y_train)
    p = model.predict_proba(X_test)[:, 1]           # predicted risk for each test patient
    test_scores[name] = p

    top20 = p >= np.quantile(p, 0.80)                # the 20% the model thinks are riskiest
    metrics[name] = {
        "AUC": round(roc_auc_score(y_test, p), 3),
        "readmissions_caught_in_top_20pct": round(y_test[top20].sum() / y_test.sum(), 3),
        "readmit_rate_in_top_20pct": round(y_test[top20].mean(), 3),
        "overall_readmit_rate": round(y_test.mean(), 3),
    }

pd.DataFrame(metrics).T

	AUC	readmissions_caught_in_top_20pct	readmit_rate_in_top_20pct	overall_readmit_rate
logistic_regression	0.629	0.352	0.158	0.09
gradient_boosting	0.630	0.355	0.159	0.09

# left: ROC curve for both models. right: the biggest logistic regression odds ratios
fig, axes = plt.subplots(1, 2, figsize=(13, 4.8))

for name, p in test_scores.items():
    fpr, tpr, _ = roc_curve(y_test, p)
    axes[0].plot(fpr, tpr, label=f"{name.replace('_', ' ')} (AUC = {metrics[name]['AUC']})")

axes[0].plot([0, 1], [0, 1], "k--", alpha=0.4, label="random guess")
axes[0].set(title="ROC curve (test set)", xlabel="False positive rate", ylabel="True positive rate")
axes[0].legend(loc="lower right")

# odds ratio = e^coefficient. above 1 raises risk, below 1 lowers it
coefs = pd.Series(models["logistic_regression"].coef_[0], index=X_train.columns)
top = coefs.reindex(coefs.abs().sort_values(ascending=False).index[:15])[::-1]
axes[1].barh(top.index, np.exp(top.values), color=np.where(top.values > 0, "#b8255f", "#96c3eb"))
axes[1].axvline(1, color="k")
axes[1].set(title="Logistic regression: top 15 odds ratios\n(red = raises risk, blue = lowers risk)", xlabel="Odds ratio")
plt.tight_layout(); plt.show()

png

Traditional ML: gives a risk score per patient and a set of coefficients. Readmission is famously hard to predict (published models on this dataset reach AUC ≈ 0.65–0.70), so the model is useful for ranking, attributes that are likely to get someone readmitted, but not as a perfect prediction model.

Limitations of ML: answers to the questions a quality committee actually asks, such as “Why is this patient high-risk?”, “Does testing HbA1c matter?” or “What should we do about it?” Real word questions. A data scientist has to write new code for every one of those questions. That is the gap an AI agent can fill.

3. Building the AI agent

3a. Tools: the agent’s “hands”

Tools are ordinary Python functions. The agent never sees our raw data or runs arbitrary code. It can only call these vetted functions, which is an important safety and privacy design choice in healthcare.

# setup the tools need: test patients with their risk scores + the logistic regression weights
test_patients = patients.iloc[idx_test].copy()
test_patients["risk_score"] = test_scores["gradient_boosting"]
test_patients["risk_percentile"] = test_patients["risk_score"].rank(pct=True).mul(100).round(1)

lr_coefs = pd.Series(models["logistic_regression"].coef_[0], index=X_train.columns)
_importance_cache = {}   # so the slow importance calc only runs once

TOOL 1: describe_dataset

# quick overview of the cleaned data: size, cleaning steps, readmission rate, features + their values
# the agent usually calls this first to get its bearings
def describe_dataset():
    return {
        "raw_encounters": int(len(raw)),
        "patients_after_cleaning": int(len(patients)),
        "cleaning_steps": ["removed deaths/hospice discharges", "kept first encounter per patient",
                           "dropped weight & payer_code (mostly missing)", "grouped ICD-9 codes into categories"],
        "target": "readmit_30 = 1 if readmitted within 30 days",
        "overall_30day_readmission_rate": round(float(patients["readmit_30"].mean()), 4),
        "numeric_features": NUMERIC,
        "categorical_features": {c: sorted(patients[c].unique().tolist()) for c in CATEGORICAL},
        "train_size": int(len(idx_train)), "test_size": int(len(idx_test)),
    }

TOOL 2: readmission_rate_by

# readmission rate for each group of ONE feature, like "discharge_to" -> Home 7%, SNF 13%...
# numeric features get split into bins first, otherwise every value would be its own group
def readmission_rate_by(feature):
    col = patients[feature]

    if feature in NUMERIC and col.nunique() > 8:
        binned = pd.qcut(col, q=5, duplicates="drop")
        if binned.nunique() >= 3:          # evenly spread values -> 5 equal-sized bins
            col = binned
        else:                              # skewed counts (mostly 0) -> 0, 1, 2, ..., "N+"
            cap = max(int(col.quantile(0.97)), 2)
            col = col.clip(upper=cap)

    g = patients.groupby(col, observed=True)["readmit_30"].agg(patients="count", readmission_rate="mean")

    # make the group labels readable
    def label(v):
        if isinstance(v, pd.Interval):     # (9.0, 13.0] -> "10-13"
            return f"{int(np.floor(v.left)) + 1}-{int(v.right)}"
        if feature in NUMERIC and v == g.index.max() and patients[feature].max() > v:
            return f"{v}+"
        return str(v)

    g.index = [label(v) for v in g.index]
    g["readmission_rate"] = g["readmission_rate"].round(4)
    return {"feature": feature, "overall_rate": round(float(patients["readmit_30"].mean()), 4),
            "groups": g.rename_axis("group").reset_index().to_dict(orient="records")}

TOOL 3: evaluate_model

# hands the agent the test-set metrics we already calculated in section 2
def evaluate_model(model_type):
    return {"model": model_type, **metrics[model_type],
            "note": "AUC 0.5 = random, 1.0 = perfect. Published models on this data reach ~0.65-0.70."}

TOOL 4: get_feature_importance

# which features matter most?
#   logistic regression -> odds ratios (tells you the DIRECTION: raises or lowers risk)
#   gradient boosting   -> permutation importance (tells you STRENGTH): shuffle a feature,
#                          and if the AUC drops a lot, the model was relying on it
def get_feature_importance(model_type, top_n):
    if model_type == "logistic_regression":
        top = lr_coefs.reindex(lr_coefs.abs().sort_values(ascending=False).index[:top_n])
        return {"model": model_type, "interpretation": "odds ratio >1 raises risk, <1 lowers it "
                "(numeric features: per 1 standard deviation increase)",
                "features": [{"feature": f, "odds_ratio": round(float(np.exp(c)), 3)} for f, c in top.items()]}

    if "gb" not in _importance_cache:
        rng = np.random.default_rng(RANDOM_STATE)
        Xs = X_test.sample(n=min(8000, len(X_test)), random_state=RANDOM_STATE)
        ys = y_test[X_test.index.get_indexer(Xs.index)]
        base = roc_auc_score(ys, models["gradient_boosting"].predict_proba(Xs)[:, 1])

        drops = {}
        for f in FEATURES:
            # shuffle all of a feature's one-hot columns together
            cols = [c for c in Xs.columns if c == f or c.startswith(f + "_")]
            Xp = Xs.copy()
            Xp[cols] = Xp[cols].values[rng.permutation(len(Xp))]
            drops[f] = base - roc_auc_score(ys, models["gradient_boosting"].predict_proba(Xp)[:, 1])
        _importance_cache["gb"] = pd.Series(drops).sort_values(ascending=False)

    top = _importance_cache["gb"].head(top_n)
    return {"model": model_type, "interpretation": "AUC drop when the feature is shuffled (bigger = more important)",
            "features": [{"feature": f, "auc_drop": round(float(v), 4)} for f, v in top.items()]}

TOOL 5: list_high_risk_patients

# the n test-set patients with the highest predicted risk (from gradient boosting)
def list_high_risk_patients(n):
    cols = ["encounter_id", "risk_score", "age_years", "primary_diagnosis", "number_inpatient", "discharge_to", "readmit_30"]
    top = test_patients.nlargest(n, "risk_score")[cols].round({"risk_score": 3})
    return {"patients": top.to_dict(orient="records"),
            "note": "readmit_30 is the actual outcome (known only because this is historical test data)"}

TOOL 6: get_patient_profile

# everything about ONE patient: their record, risk score, and WHY they're high/low risk
# "why" = each feature's contribution = coefficient x (this patient's value - the average patient)
def get_patient_profile(encounter_id):
    match = test_patients[test_patients["encounter_id"] == encounter_id]
    if match.empty:
        raise ValueError(f"encounter_id {encounter_id} not found in the test set")   # agent sees this and can retry

    row = match.iloc[0]
    x = X_test.loc[match.index[0]]
    contrib = (lr_coefs * (x - X_train.mean())).sort_values()

    # a 0/1 column the patient does NOT have gets labelled "NOT ..." so it's clear
    contrib.index = [f if f in NUMERIC or x[f] == 1 else f"NOT {f}" for f in contrib.index]

    return {
        "encounter_id": int(encounter_id),
        "risk_score": round(float(row["risk_score"]), 3),
        "risk_percentile": float(row["risk_percentile"]),
        "baseline_readmission_rate": round(float(patients["readmit_30"].mean()), 3),
        "record": {f: (row[f].item() if hasattr(row[f], "item") else row[f]) for f in FEATURES},
        "top_factors_increasing_risk": [{"feature": f, "contribution": round(float(v), 3)} for f, v in contrib[::-1][:6].items()],
        "top_factors_decreasing_risk": [{"feature": f, "contribution": round(float(v), 3)} for f, v in contrib[:3].items()],
        "actual_outcome_readmitted_30d": int(row["readmit_30"]),
    }

TOOL 7: plot_readmission_rate

# lets the agent make its own bar chart (uses tool 2 for the numbers), saves it to outputs/
def plot_readmission_rate(feature):
    data = readmission_rate_by(feature)
    g = pd.DataFrame(data["groups"])

    fig, ax = plt.subplots(figsize=(8, 4))
    ax.bar(g["group"].astype(str), g["readmission_rate"] * 100, color="#96c3eb")
    ax.axhline(data["overall_rate"] * 100, color="#b8255f", ls="--", label=f"overall {data['overall_rate']:.1%}")
    ax.set(title=f"30-day readmission rate by {feature}", ylabel="Readmission rate (%)")
    ax.legend(); plt.xticks(rotation=30, ha="right"); plt.tight_layout()

    path = OUTPUT_DIR / f"readmission_by_{feature}.png"
    fig.savefig(path); plt.show()
    return {"chart_saved_to": str(path), **data}

# the agent asks for a tool by name -> this dictionary finds the matching python function
TOOL_FUNCTIONS = {f.__name__: f for f in [describe_dataset, readmission_rate_by, evaluate_model,
                                          get_feature_importance, list_high_risk_patients,
                                          get_patient_profile, plot_readmission_rate]}

# quick test that a tool works on its own (no AI involved yet):
readmission_rate_by("number_inpatient")

{'feature': 'number_inpatient',
 'overall_rate': 0.0898,
 'groups': [{'group': '0', 'patients': 61792, 'readmission_rate': 0.0813},
  {'group': '1', 'patients': 5798, 'readmission_rate': 0.1288},
  {'group': '2+', 'patients': 2397, 'readmission_rate': 0.2149}]}

3b. Tool definitions: how the LLM learns what each tool does

The model only sees each tool’s name, description and input schema (in JSON Schema format). Clear descriptions matter: they are how the agent decides which tool fits. enum lists restrict arguments to valid values, and strict: True guarantees the model’s arguments match the schema.

# helper so we don't have to write out the full JSON schema 7 times
# strict=True -> the model's arguments are guaranteed to match the schema
def tool(name, description, properties=None):
    properties = properties or {}
    return {"name": name, "description": description, "strict": True,
            "input_schema": {"type": "object", "properties": properties,
                             "required": list(properties), "additionalProperties": False}}

# the "menu" the agent picks from. the descriptions are basically instructions,
# it's how the model decides which tool fits the question
# enum = only these values are allowed (so it can't ask for a column that doesn't exist)
MODEL_ENUM = {"type": "string", "enum": ["logistic_regression", "gradient_boosting"]}

TOOLS = [
    tool("describe_dataset", "Summary of the cleaned diabetic-patient dataset: size, cleaning steps, "
         "overall 30-day readmission rate, and the list of features with their possible values. Call this first."),
    tool("readmission_rate_by", "30-day readmission rate broken down by the groups of ONE feature "
         "(numeric features are split into quintiles). Use it to explore associations.",
         {"feature": {"type": "string", "enum": FEATURES}}),
    tool("evaluate_model", "Test-set performance (AUC, share of readmissions caught in the top-20% riskiest "
         "patients) of an already-trained model.", {"model_type": MODEL_ENUM}),
    tool("get_feature_importance", "Most important features of a trained model. Logistic regression returns "
         "odds ratios (direction of effect); gradient boosting returns permutation importance (strength of effect).",
         {"model_type": MODEL_ENUM, "top_n": {"type": "integer", "description": "between 3 and 20"}}),
    tool("list_high_risk_patients", "The n patients in the held-out test set with the highest predicted "
         "30-day readmission risk.", {"n": {"type": "integer", "description": "between 1 and 20"}}),
    tool("get_patient_profile", "Full record of one test-set patient: predicted risk, percentile, and the "
         "specific factors raising or lowering their risk.", {"encounter_id": {"type": "integer"}}),
    tool("plot_readmission_rate", "Create and display a bar chart of 30-day readmission rate by one feature. "
         "Also returns the underlying numbers.", {"feature": {"type": "string", "enum": FEATURES}}),
]

# this is what the model sees for one tool
print(json.dumps(TOOLS[1], indent=2))

{
  "name": "readmission_rate_by",
  "description": "30-day readmission rate broken down by the groups of ONE feature (numeric features are split into quintiles). Use it to explore associations.",
  "strict": true,
  "input_schema": {
    "type": "object",
    "properties": {
      "feature": {
        "type": "string",
        "enum": [
          "age_years",
          "time_in_hospital",
          "num_lab_procedures",
          "num_procedures",
          "num_medications",
          "number_outpatient",
          "number_emergency",
          "number_inpatient",
          "number_diagnoses",
          "num_diabetes_meds",
          "gender",
          "race",
          "admission_type",
          "admission_source",
          "discharge_to",
          "primary_diagnosis",
          "medical_specialty",
          "A1C_result",
          "max_glucose",
          "insulin",
          "med_change",
          "on_diabetes_med"
        ]
      }
    },
    "required": [
      "feature"
    ],
    "additionalProperties": false
  }
}

3c. The model and the agent loop

Call the LLM -> if it asks for tools, run them and send back results -> repeat until it answers.
The loop also enforces guardrails: a maximum number of steps, error handling, and a running cost tracker.

The API key is read from the ANTHROPIC_API_KEY environment variable, or from a local .env file. Or it’s pasted into the console (but never saved)

# load the API key: environment variable first, then a .env file (ANTHROPIC_API_KEY=sk-ant-...)
# if neither is found, it asks you to paste it
if not os.environ.get("ANTHROPIC_API_KEY") and Path(".env").exists():
    for line in Path(".env").read_text().splitlines():
        if line.strip().startswith("ANTHROPIC_API_KEY="):
            os.environ["ANTHROPIC_API_KEY"] = line.split("=", 1)[1].strip().strip('"').strip("'")

if not os.environ.get("ANTHROPIC_API_KEY"):
    from getpass import getpass
    os.environ["ANTHROPIC_API_KEY"] = getpass("Paste your Anthropic API key: ")

# connect to Claude and pick the model
client = anthropic.Anthropic()
MODEL = "claude-opus-5-5"

# prices in USD per million tokens, so we can track what each run costs
PRICE_PER_MTOK = {"input": 4.00, "output": 20.00, "cache_write": 5.00, "cache_read": 0.20}

# system prompt = the agent's job description + rules. it gets sent with every request
SYSTEM_PROMPT = '''You are a clinical data-analyst agent supporting a hospital quality-improvement team
that wants to reduce 30-day readmissions among diabetic patients (a Medicare HRRP penalty measure).

You have tools that query a cleaned dataset of ~70,000 diabetic inpatients from 130 US hospitals (1999-2008)
and two already-trained models. Work like a careful analyst:
- Ground every number you report in a tool result. Never invent statistics.
- Use several tools when needed: explore the data, check model evidence, and cross-check findings.
- Remember that associations are not causation, and note important caveats.
- Your final answer is read by clinicians and administrators: clear plain language, under ~300 words,
  use short headings/bullets, and end with concrete, practical recommendations.
- You are decision SUPPORT; final clinical decisions belong to the care team.'''

# runs whichever tool the agent asked for
# if the tool breaks, we send the error back to the agent instead of crashing, so it can try something else
def execute_tool(name, args):
    try:
        result = TOOL_FUNCTIONS[name](**args)
        return json.dumps(result, default=str), False
    except Exception as e:
        return f"Error: {e}", True

# THE AGENT LOOP (ReAct): think -> pick a tool -> see the result -> repeat until it has an answer
# the 4 building blocks are labelled: MODEL, TOOLS, MEMORY, GUARDRAIL
def run_agent(question, max_steps=12):
    print("=" * 100 + f"\nGOAL: {question}\n" + "=" * 100)
    messages = [{"role": "user", "content": question}]   # <- MEMORY: the growing conversation
    cost, n_tool_calls = 0.0, 0

    for step in range(1, max_steps + 1):                   # <- GUARDRAIL: can't loop forever
        response = client.beta.messages.create(            # <- MODEL: the LLM "brain"
            model=MODEL,
            max_tokens=16000,
            system=SYSTEM_PROMPT,
            tools=TOOLS,                                   # <- TOOLS it can choose from
            messages=messages,
            thinking={"type": "adaptive", "display": "summarized"},  # show a summary of its reasoning
            output_config={"effort": "medium"},
            cache_control={"type": "ephemeral"},           # reuse the parts that didn't change -> cheaper
            betas=["server-side-fallback-2026-07-01"],
            fallbacks="default",                           # if a request gets declined, retry on a backup model
        )

        # add up the cost of this step
        u = response.usage
        cost += (u.input_tokens * PRICE_PER_MTOK["input"] + u.output_tokens * PRICE_PER_MTOK["output"]
                 + (u.cache_creation_input_tokens or 0) * PRICE_PER_MTOK["cache_write"]
                 + (u.cache_read_input_tokens or 0) * PRICE_PER_MTOK["cache_read"]) / 1e6

        if response.stop_reason == "refusal":
            print("The model declined this request."); return None
        messages.append({"role": "assistant", "content": response.content})

        # print what the agent is thinking and which tools it picked
        print(f"\n--- Step {step} ---")
        for block in response.content:
            if block.type == "thinking" and block.thinking.strip():
                print(f"[THINK]  {block.thinking.strip()[:600]}")
            elif block.type == "tool_use":
                print(f"[ACT]    {block.name}({json.dumps(block.input)})")

        # no tool requested = it's done, return the final answer
        if response.stop_reason != "tool_use":
            answer = "\n".join(b.text for b in response.content if b.type == "text")
            print(f"\n{'=' * 100}\nFINAL ANSWER  ({step} steps, {n_tool_calls} tool calls, "
                  f"API cost ≈ ${cost:.3f})\n{'=' * 100}\n")
            return answer

        # otherwise run every tool it asked for, and send all the results back together
        tool_results = []
        for block in response.content:
            if block.type == "tool_use":
                result, is_error = execute_tool(block.name, block.input)
                n_tool_calls += 1
                print(f"[OBSERVE] {result[:250]}{'...' if len(result) > 250 else ''}")
                tool_results.append({"type": "tool_result", "tool_use_id": block.id,
                                     "content": result, "is_error": is_error})
        messages.append({"role": "user", "content": tool_results})

    print(f"Stopped: reached max_steps={max_steps} (guardrail). API cost ≈ ${cost:.3f}")
    return None

4. Agent demos

Watch!! [THINK] = the agent’s reasoning, [ACT] = the tool it chose, [OBSERVE] = what the tool returned. Agent was never told which tools to use or in what order.

Demo 1: Open-ended investigation for the quality committee

# lets us show the agent's final answer as nicely formatted text instead of raw markdown
from IPython.display import Markdown, display

# demo 1: big open-ended question, we don't tell it HOW to investigate
answer = run_agent(
    "Our hospital is being penalized by Medicare for 30-day readmissions among diabetic patients. "
    "Investigate which patient and hospital-stay characteristics are most strongly associated with "
    "30-day readmission, and tell us how reliably we can predict it. Summarize the key drivers for "
    "our quality committee."
)
display(Markdown(answer))

====================================================================================================
GOAL: Our hospital is being penalized by Medicare for 30-day readmissions among diabetic patients. Investigate which patient and hospital-stay characteristics are most strongly associated with 30-day readmission, and tell us how reliably we can predict it. Summarize the key drivers for our quality committee.
====================================================================================================

--- Step 1 ---
[THINK]  I should start by getting a sense of what's in the dataset.
[ACT]    describe_dataset({})
[ACT]    evaluate_model({"model_type": "logistic_regression"})
[ACT]    evaluate_model({"model_type": "gradient_boosting"})
[ACT]    get_feature_importance({"model_type": "gradient_boosting", "top_n": 10})
[ACT]    get_feature_importance({"model_type": "logistic_regression", "top_n": 10})
[OBSERVE] {"raw_encounters": 101766, "patients_after_cleaning": 69987, "cleaning_steps": ["removed deaths/hospice discharges", "kept first encounter per patient", "dropped weight & payer_code (mostly missing)", "grouped ICD-9 codes into categories"], "target":...
[OBSERVE] {"model": "logistic_regression", "AUC": 0.629, "readmissions_caught_in_top_20pct": 0.352, "readmit_rate_in_top_20pct": 0.158, "overall_readmit_rate": 0.09, "note": "AUC 0.5 = random, 1.0 = perfect. Published models on this data reach ~0.65-0.70."}
[OBSERVE] {"model": "gradient_boosting", "AUC": 0.63, "readmissions_caught_in_top_20pct": 0.355, "readmit_rate_in_top_20pct": 0.159, "overall_readmit_rate": 0.09, "note": "AUC 0.5 = random, 1.0 = perfect. Published models on this data reach ~0.65-0.70."}
[OBSERVE] {"model": "gradient_boosting", "interpretation": "AUC drop when the feature is shuffled (bigger = more important)", "features": [{"feature": "discharge_to", "auc_drop": 0.0694}, {"feature": "number_inpatient", "auc_drop": 0.018}, {"feature": "primary...
[OBSERVE] {"model": "logistic_regression", "interpretation": "odds ratio >1 raises risk, <1 lowers it (numeric features: per 1 standard deviation increase)", "features": [{"feature": "discharge_to_Other inpatient / rehab facility", "odds_ratio": 2.099}, {"feat...

--- Step 2 ---
[ACT]    readmission_rate_by({"feature": "discharge_to"})
[ACT]    readmission_rate_by({"feature": "number_inpatient"})
[ACT]    readmission_rate_by({"feature": "primary_diagnosis"})
[ACT]    readmission_rate_by({"feature": "age_years"})
[ACT]    readmission_rate_by({"feature": "A1C_result"})
[ACT]    readmission_rate_by({"feature": "time_in_hospital"})
[OBSERVE] {"feature": "discharge_to", "overall_rate": 0.0898, "groups": [{"group": "Home", "patients": 44320, "readmission_rate": 0.0695}, {"group": "Home with health services", "patients": 8364, "readmission_rate": 0.0954}, {"group": "Left against medical adv...
[OBSERVE] {"feature": "number_inpatient", "overall_rate": 0.0898, "groups": [{"group": "0", "patients": 61792, "readmission_rate": 0.0813}, {"group": "1", "patients": 5798, "readmission_rate": 0.1288}, {"group": "2+", "patients": 2397, "readmission_rate": 0.21...
[OBSERVE] {"feature": "primary_diagnosis", "overall_rate": 0.0898, "groups": [{"group": "Circulatory", "patients": 21389, "readmission_rate": 0.0968}, {"group": "Diabetes", "patients": 5748, "readmission_rate": 0.0912}, {"group": "Digestive", "patients": 6488,...
[OBSERVE] {"feature": "age_years", "overall_rate": 0.0898, "groups": [{"group": "5-55", "patients": 23679, "readmission_rate": 0.0712}, {"group": "56-65", "patients": 15688, "readmission_rate": 0.0902}, {"group": "66-75", "patients": 17749, "readmission_rate":...
[OBSERVE] {"feature": "A1C_result", "overall_rate": 0.0898, "groups": [{"group": ">7", "patients": 2866, "readmission_rate": 0.0862}, {"group": ">8", "patients": 6239, "readmission_rate": 0.0816}, {"group": "Norm", "patients": 3741, "readmission_rate": 0.0863}...
[OBSERVE] {"feature": "time_in_hospital", "overall_rate": 0.0898, "groups": [{"group": "1-2", "patients": 22651, "readmission_rate": 0.0716}, {"group": "3-3", "patients": 12493, "readmission_rate": 0.0821}, {"group": "4-4", "patients": 9423, "readmission_rate"...

--- Step 3 ---
[THINK]  This is good enough to write up. I should note the dataset spans 1999-2008 across 130 hospitals, death/hospice cases were excluded, only first encounters are included, A1C wasn't tested in 82% of cases, and discharge disposition is heavily confounded by illness severity.

====================================================================================================
FINAL ANSWER  (3 steps, 11 tool calls, API cost ≈ $0.065)
====================================================================================================

30-Day Readmission Drivers: Diabetic Inpatients

Overall rate: 9.0% of 69,987 patients were readmitted within 30 days.

Strongest drivers (both models agree, and the raw rates confirm it)

Where the patient goes at discharge is by far the strongest signal.
- Home: 7.0% readmitted
- Home with health services: 9.5%
- Skilled nursing / long-term care: 13.1%
- Other inpatient / rehab facility: 20.3%
Prior inpatient stays in the past year:
- None: 8.1%
- One: 12.9%
- Two or more: 21.5%
Longer stays: 1–2 days: 7.2%; 7–14 days: 11.7%.
Older age: under 56: 7.1%; 76 and older: 10.6%.
Primary diagnosis matters a little: injury 10.8%, circulatory 9.7%, respiratory 7.3%.

Weak signal: the A1C result barely differs between groups (8.2–9.1%). But 82% of patients were never tested, which itself points to a gap in care.

How well can we predict it?

Only modestly. Both models scored an AUC of about 0.63, where 0.5 is a coin flip and 1.0 is perfect. Published models on this data reach about 0.65–0.70. - The 20% of patients rated highest-risk had a 16% readmission rate, compared with 9% overall. - That group contains only about 35% of all readmissions, so most readmissions happen in patients rated lower-risk. - The model is useful for deciding who gets extra attention first. It is not reliable enough to make decisions about individual patients.

Caveats

These are associations, not causes. Discharge to a facility most likely reflects how sick and frail the patient is, not poor care at the facility.
The data comes from 130 US hospitals in 1999–2008, not from our hospital. Patients who died or went to hospice were excluded, and only each patient’s first stay was counted.

Recommendations

Flag high-risk patients at admission: anyone with 1 or more inpatient stays in the past year, a stay longer than 5 days, or age over 65.
Strengthen handoffs to facilities: send a structured medication and care summary to skilled nursing and rehab facilities, and follow up by phone within 48–72 hours.
Prioritize the top-risk group for teach-back education, a follow-up visit within 7 days, and medication reconciliation.
Close the A1C gap: test A1C routinely during admission so diabetes medications can be adjusted before discharge.
Validate on our own data before relying on any risk score, and measure whether these steps actually lower our readmission rate.

These tools support decisions; final clinical decisions stay with the care team.

[OBSERVE] {"model": "gradient_boosting", "interpretation": "AUC drop when the feature is shuffled (bigger = more important)", "features": [{"feature": "discharge_to", "auc_drop": 0.0798}, {"feature": "number_inpatient", "auc_drop": 0.0382}, {"feature": "age_ye...
[OBSERVE] {"model": "logistic_regression", "interpretation": "odds ratio >1 raises risk, <1 lowers it (numeric features: per 1 standard deviation increase)", "features": [{"feature": "discharge_to_Other inpatient / rehab facility", "odds_ratio": 2.003}, {"feat...



--- Step 2 ---
[ACT]    readmission_rate_by({"feature": "discharge_to"})
[ACT]    readmission_rate_by({"feature": "number_inpatient"})
[ACT]    readmission_rate_by({"feature": "age_years"})
[ACT]    readmission_rate_by({"feature": "A1C_result"})
[ACT]    readmission_rate_by({"feature": "primary_diagnosis"})
[OBSERVE] {"feature": "discharge_to", "overall_rate": 0.0898, "groups": [{"group": "Home", "patients": 44320, "readmission_rate": 0.0695}, {"group": "Home with health services", "patients": 8364, "readmission_rate": 0.0954}, {"group": "Left against medical adv...
[OBSERVE] {"feature": "number_inpatient", "overall_rate": 0.0898, "groups": [{"group": "0", "patients": 61792, "readmission_rate": 0.0813}, {"group": "1", "patients": 5798, "readmission_rate": 0.1288}, {"group": "2+", "patients": 2397, "readmission_rate": 0.21...
[OBSERVE] {"feature": "age_years", "overall_rate": 0.0898, "groups": [{"group": "5-55", "patients": 23679, "readmission_rate": 0.0712}, {"group": "56-65", "patients": 15688, "readmission_rate": 0.0902}, {"group": "66-75", "patients": 17749, "readmission_rate":...
[OBSERVE] {"feature": "A1C_result", "overall_rate": 0.0898, "groups": [{"group": ">7", "patients": 2866, "readmission_rate": 0.0862}, {"group": ">8", "patients": 6239, "readmission_rate": 0.0816}, {"group": "Norm", "patients": 3741, "readmission_rate": 0.0863}...
[OBSERVE] {"feature": "primary_diagnosis", "overall_rate": 0.0898, "groups": [{"group": "Circulatory", "patients": 21389, "readmission_rate": 0.0968}, {"group": "Diabetes", "patients": 5748, "readmission_rate": 0.0912}, {"group": "Digestive", "patients": 6488,...



--- Step 3 ---
[THINK]  I should also glance at time_in_hospital and number_emergency for a couple more checks.
[ACT]    readmission_rate_by({"feature": "time_in_hospital"})
[ACT]    readmission_rate_by({"feature": "number_emergency"})
[OBSERVE] {"feature": "time_in_hospital", "overall_rate": 0.0898, "groups": [{"group": "1-2", "patients": 22651, "readmission_rate": 0.0716}, {"group": "3-3", "patients": 12493, "readmission_rate": 0.0821}, {"group": "4-4", "patients": 9423, "readmission_rate"...
[OBSERVE] {"feature": "number_emergency", "overall_rate": 0.0898, "groups": [{"group": "0", "patients": 64886, "readmission_rate": 0.0874}, {"group": "1", "patients": 3883, "readmission_rate": 0.1125}, {"group": "2+", "patients": 1218, "readmission_rate": 0.14...



--- Step 4 ---
[THINK]  I should keep the summary concise, under 300 words.

====================================================================================================
FINAL ANSWER  (4 steps, 12 tool calls, API cost ≈ $0.068)
====================================================================================================

Diabetic 30-Day Readmissions: Key Drivers and How Well We Can Predict Them

Data: 69,987 diabetic patients from 130 US hospitals, 1999–2008. Each patient’s first stay only; patients who died or went to hospice were removed. Overall, 9.0% were readmitted within 30 days. These are not our own hospital’s patients, so local results may differ.

Strongest drivers (both models agree, and the raw rates confirm it)

Where the patient goes at discharge. This is by far the strongest factor.
- Sent to another inpatient or rehab facility: 20.3% readmitted
- Skilled nursing or long-term care: 13.1%
- Home with health services: 9.5%
- Home: 7.0%
Hospital stays in the past year. No prior stays: 8.1%. One: 12.9%. Two or more: 21.5%.
Prior emergency visits. None: 8.7%. Two or more: 14.5%.
Longer stay. 1–2 days: 7.2%. 7–14 days: 11.7%.
Older age. Under 56: 7.1%. 76 and over: 10.6%.
Main diagnosis and specialty matter a little. Injury (10.8%) and circulatory (9.7%) are slightly higher; respiratory is lower (7.3%).

Weak or no link: HbA1c results. Most patients (57,141) were never tested, and rates were similar across all groups (8.2–9.1%).

How reliable is prediction?

Only moderately reliable. The best model scores 0.65 on a 0.5–1.0 scale, where 0.5 is a coin flip. That is in line with published models on this data.
The 20% of patients it flags as highest risk are readmitted at 17.7%, about twice the average. But that group contains only 39% of all readmissions, so most readmissions happen in patients the model ranks lower.
Use it to rank patients for extra help, not to make individual decisions.

Caveats

These are associations, not proven causes. Patients sent to facilities are probably sicker to begin with.
The data is old (1999–2008), and important factors are missing, such as social circumstances, follow-up appointments and medication adherence.

Recommendations

Focus on handoffs to other facilities: medication reconciliation, a direct call between clinicians, and a shared care plan for patients going to rehab or nursing facilities.
Flag patients with frequent recent use (prior admissions or ER visits) when they’re admitted for case management and a follow-up visit within 7 days.
Add extra discharge planning for older patients and stays of 5 days or more.
Retrain and check the model on our own recent data before using it. Treat it as a way to prioritize, alongside clinical judgment. Final decisions stay with the care team.

Demo 2: Testing a published claim

The paper that released this dataset (Strack et al., 2014) argued that measuring HbA1c (a blood test for long-term glucose control) during a hospital stay is associated with lower readmission. Let’s ask the agent to check this, including making its own chart.

# demo 2: fact-check a published claim + make a chart on its own
answer = run_agent(
    "A 2014 study claimed that measuring HbA1c during the hospital stay is associated with lower "
    "readmission rates. Does our data support that? Make a chart, and consider whether other factors "
    "could explain the pattern."
)
display(Markdown(answer))

====================================================================================================
GOAL: A 2014 study claimed that measuring HbA1c during the hospital stay is associated with lower readmission rates. Does our data support that? Make a chart, and consider whether other factors could explain the pattern.
====================================================================================================

--- Step 1 ---
[ACT]    describe_dataset({})
[ACT]    plot_readmission_rate({"feature": "A1C_result"})
[OBSERVE] {"raw_encounters": 101766, "patients_after_cleaning": 69987, "cleaning_steps": ["removed deaths/hospice discharges", "kept first encounter per patient", "dropped weight & payer_code (mostly missing)", "grouped ICD-9 codes into categories"], "target":...

png

[OBSERVE] {"chart_saved_to": "outputs/readmission_by_A1C_result.png", "feature": "A1C_result", "overall_rate": 0.0898, "groups": [{"group": ">7", "patients": 2866, "readmission_rate": 0.0862}, {"group": ">8", "patients": 6239, "readmission_rate": 0.0816}, {"gr...

--- Step 2 ---
[THINK]  The tested group shows readmission rates around 8.2-8.6% compared to 9.1% for untested patients—a small gap. I want to check for confounders like admission type, diagnosis, specialty, and prior inpatient visits, since I can't easily cross-tabulate here, so I'll look at logistic regression odds ratios and gradient boosting feature importance instead.
[ACT]    get_feature_importance({"model_type": "logistic_regression", "top_n": 20})
[ACT]    get_feature_importance({"model_type": "gradient_boosting", "top_n": 15})
[ACT]    readmission_rate_by({"feature": "primary_diagnosis"})
[ACT]    readmission_rate_by({"feature": "admission_type"})
[ACT]    readmission_rate_by({"feature": "medical_specialty"})
[ACT]    readmission_rate_by({"feature": "med_change"})
[OBSERVE] {"model": "logistic_regression", "interpretation": "odds ratio >1 raises risk, <1 lowers it (numeric features: per 1 standard deviation increase)", "features": [{"feature": "discharge_to_Other inpatient / rehab facility", "odds_ratio": 2.003}, {"feat...
[OBSERVE] {"model": "gradient_boosting", "interpretation": "AUC drop when the feature is shuffled (bigger = more important)", "features": [{"feature": "discharge_to", "auc_drop": 0.0798}, {"feature": "number_inpatient", "auc_drop": 0.0382}, {"feature": "age_ye...
[OBSERVE] {"feature": "primary_diagnosis", "overall_rate": 0.0898, "groups": [{"group": "Circulatory", "patients": 21389, "readmission_rate": 0.0968}, {"group": "Diabetes", "patients": 5748, "readmission_rate": 0.0912}, {"group": "Digestive", "patients": 6488,...
[OBSERVE] {"feature": "admission_type", "overall_rate": 0.0898, "groups": [{"group": "Elective", "patients": 13786, "readmission_rate": 0.0828}, {"group": "Emergency", "patients": 35478, "readmission_rate": 0.0918}, {"group": "Other/Unknown", "patients": 7920,...
[OBSERVE] {"feature": "medical_specialty", "overall_rate": 0.0898, "groups": [{"group": "Cardiology", "patients": 4207, "readmission_rate": 0.072}, {"group": "Emergency/Trauma", "patients": 4393, "readmission_rate": 0.0783}, {"group": "Family/GeneralPractice",...
[OBSERVE] {"feature": "med_change", "overall_rate": 0.0898, "groups": [{"group": "No", "patients": 38492, "readmission_rate": 0.086}, {"group": "Yes", "patients": 31495, "readmission_rate": 0.0944}]}

--- Step 3 ---
[THINK]  Computing the pooled rate for tested patients gives about 8.40% versus 9.11% for untested, a modest ~0.7 point gap. A1C doesn't rank in the top factors for logistic regression and barely moves the AUC in gradient boosting, so it seems negligible once other variables are accounted for, though a direct stratified analysis isn't feasible here.

Strack et al.'s 2014 study found this association actually depends heavily on primary diagnosis, and only 18% of patients got tested here, so confounders like which specialty or hospital ordered the test likely matter a lot. Since I can't cross-tabulate di

====================================================================================================
FINAL ANSWER  (3 steps, 8 tool calls, API cost ≈ $0.064)
====================================================================================================

Short answer: only a weak signal, and it may not mean much

Chart: saved to outputs/readmission_by_A1C_result.png

HbA1c status	Patients	30-day readmission
Not tested	57,141	9.1%
Normal	3,741	8.6%
>7	2,866	8.6%
>8	6,239	8.2%

Patients who had an HbA1c test were readmitted at about 8.4%, compared with 9.1% for those who weren’t. That’s about 0.7 percentage points lower. (I worked out the 8.4% by combining the three tested groups.)
The direction matches the 2014 study, but the gap is small. Only about 18% of patients were tested at all.

Other factors explain most of it

The models barely use HbA1c. In the logistic regression, HbA1c isn’t among the 20 strongest factors. In the gradient-boosting model it ranks 15th, and removing it changes accuracy by almost nothing (AUC drop 0.0009). Compare that with discharge destination (0.080) and prior inpatient visits (0.038). Once other patient details are accounted for, HbA1c testing adds little.
Who gets tested isn’t random. Readmission rates already differ by group, and these groups probably differ in how often they test:
- Primary diagnosis: respiratory 7.3% vs injury 10.8%
- Specialty: cardiology 7.2% vs internal medicine 9.8%
- Admission type: elective 8.3% vs emergency 9.2%
Testing may also be a sign of a more thorough hospital or care team, rather than something that prevents readmission by itself.
Limits: My tools can’t split the HbA1c comparison by diagnosis or hospital, so I couldn’t confirm these explanations directly. The data is from 1999–2008 and observational, so it can’t show cause and effect.

Recommendations

Don’t expect HbA1c testing alone to lower readmissions. It’s still good diabetes care, but our data doesn’t support it as a readmission fix.
Focus on the stronger factors: discharge planning (patients discharged to another inpatient or rehab facility had about twice the odds of readmission) and patients with prior admissions.
If you test, act on the result. Link high HbA1c values to medication review and follow-up after discharge. Any benefit likely comes from what’s done with the result, not the test itself.
For a fair test, compare like with like (same diagnosis, same hospital), or run a small pilot that tracks both testing and what follows.

Final clinical decisions rest with the care team.

--- Step 2 ---
[THINK]  Computing the readmission rate for the tested group across the three categories gives about 8.40%, compared to 9.11% for untested patients—a roughly 0.7 point absolute gap, around 8% relative. Now I want to check whether confounders like primary diagnosis, admission type, or medical specialty explain this difference, so I'll look at feature importance for A1C.
[ACT]    readmission_rate_by({"feature": "primary_diagnosis"})
[ACT]    readmission_rate_by({"feature": "medical_specialty"})
[ACT]    readmission_rate_by({"feature": "admission_type"})
[ACT]    get_feature_importance({"model_type": "logistic_regression", "top_n": 20})
[ACT]    get_feature_importance({"model_type": "gradient_boosting", "top_n": 20})
[OBSERVE] {"feature": "primary_diagnosis", "overall_rate": 0.0898, "groups": [{"group": "Circulatory", "patients": 21389, "readmission_rate": 0.0968}, {"group": "Diabetes", "patients": 5748, "readmission_rate": 0.0912}, {"group": "Digestive", "patients": 6488,...
[OBSERVE] {"feature": "medical_specialty", "overall_rate": 0.0898, "groups": [{"group": "Cardiology", "patients": 4207, "readmission_rate": 0.072}, {"group": "Emergency/Trauma", "patients": 4393, "readmission_rate": 0.0783}, {"group": "Family/GeneralPractice",...
[OBSERVE] {"feature": "admission_type", "overall_rate": 0.0898, "groups": [{"group": "Elective", "patients": 13786, "readmission_rate": 0.0828}, {"group": "Emergency", "patients": 35478, "readmission_rate": 0.0918}, {"group": "Other/Unknown", "patients": 7920,...
[OBSERVE] {"model": "logistic_regression", "interpretation": "odds ratio >1 raises risk, <1 lowers it (numeric features: per 1 standard deviation increase)", "features": [{"feature": "discharge_to_Other inpatient / rehab facility", "odds_ratio": 2.003}, {"feat...
[OBSERVE] {"model": "gradient_boosting", "interpretation": "AUC drop when the feature is shuffled (bigger = more important)", "features": [{"feature": "discharge_to", "auc_drop": 0.0798}, {"feature": "number_inpatient", "auc_drop": 0.0382}, {"feature": "age_ye...



--- Step 3 ---
[THINK]  Med_change could also be a mediator, since Strack et al. found the effect depends on primary diagnosis—I should quickly check med_change and discharge_to as well.
[ACT]    readmission_rate_by({"feature": "med_change"})
[ACT]    readmission_rate_by({"feature": "discharge_to"})
[OBSERVE] {"feature": "med_change", "overall_rate": 0.0898, "groups": [{"group": "No", "patients": 38492, "readmission_rate": 0.086}, {"group": "Yes", "patients": 31495, "readmission_rate": 0.0944}]}
[OBSERVE] {"feature": "discharge_to", "overall_rate": 0.0898, "groups": [{"group": "Home", "patients": 44320, "readmission_rate": 0.0695}, {"group": "Home with health services", "patients": 8364, "readmission_rate": 0.0954}, {"group": "Left against medical adv...



--- Step 4 ---
[THINK]  My tools don't let me cross-tabulate A1C by diagnosis, so I can't adjust for that directly—I should just be upfront about this limitation and keep my response concise.

====================================================================================================
FINAL ANSWER  (4 steps, 9 tool calls, API cost ≈ $0.068)
====================================================================================================

Does testing HbA1c in hospital go with fewer readmissions?

Short answer: We see a small pattern in the same direction as the study, but it’s weak. Our data can’t show that testing causes fewer readmissions.

What the chart shows

Chart saved to outputs/readmission_by_A1C_result.png.

HbA1c status	Patients	30-day readmission
Not tested	57,141	9.1%
Normal	3,741	8.6%
>7	2,866	8.6%
>8	6,239	8.2%

About 82% of patients were never tested.
Patients who were tested come back about 8.4% of the time, versus 9.1% for those who weren’t. That’s under 1 percentage point, from simple counts with no adjustment for other factors.

Other factors that could explain it

Other factors matter far more. Where patients go after discharge has the biggest effect: 7.0% readmitted if sent home, 13.1% from nursing facilities, 20.3% from rehab or another hospital. Diagnosis, specialty and type of admission also vary (for example, Cardiology 7.2% vs Internal Medicine 9.8%).
The model gives HbA1c almost no weight. It ranks 15th of 20 factors in the gradient-boosting model, and is outside the top 20 in the logistic regression.
Who gets tested isn’t random. Testing likely depends on the specialty, the hospital and how sick the patient is. Our tools can’t break testing down by those factors, so we couldn’t adjust for them.
Testing may stand for better overall care rather than having an effect of its own.

Caveats

The data is from 1999–2008 and observational. It counts readmissions to the same health system only.
We didn’t run a statistical test, so we can’t say if the gap is more than chance.

Recommendations

Keep testing HbA1c because it’s good diabetes care, not as a main way to reduce readmissions.
Put readmission effort where the signal is strongest: patients discharged to rehab or nursing facilities, and those with past hospital stays. Focus on handoffs and follow-up.
Track whether HbA1c results change treatment. Testing only helps if someone acts on the result.
Ask analytics for an adjusted analysis (by diagnosis, specialty and hospital) before choosing an HbA1c-based strategy.

Final clinical decisions belong to the care team.

Demo 3: Patient-level decision support (discharge planning)

An operational task: find the highest-risk patients and prepare a briefing a nurse or case manager could act on.

# demo 3: patient-level, the kind of thing a case manager would actually use
answer = run_agent(
    "Identify the 5 highest-risk patients in the test set. For the single highest-risk patient, prepare a "
    "short discharge-planning briefing for the care team: why this patient is high risk (based on their "
    "specific factors) and which interventions should be considered before and after discharge."
)
display(Markdown(answer))

====================================================================================================
GOAL: Identify the 5 highest-risk patients in the test set. For the single highest-risk patient, prepare a short discharge-planning briefing for the care team: why this patient is high risk (based on their specific factors) and which interventions should be considered before and after discharge.
====================================================================================================

--- Step 1 ---
[ACT]    list_high_risk_patients({"n": 5})
[ACT]    describe_dataset({})
[OBSERVE] {"patients": [{"encounter_id": 196770162, "risk_score": 0.561, "age_years": 25, "primary_diagnosis": "Diabetes", "number_inpatient": 8, "discharge_to": "Home", "readmit_30": 1}, {"encounter_id": 160455492, "risk_score": 0.51, "age_years": 75, "primar...
[OBSERVE] {"raw_encounters": 101766, "patients_after_cleaning": 69987, "cleaning_steps": ["removed deaths/hospice discharges", "kept first encounter per patient", "dropped weight & payer_code (mostly missing)", "grouped ICD-9 codes into categories"], "target":...

--- Step 2 ---
[ACT]    get_patient_profile({"encounter_id": 196770162})
[ACT]    evaluate_model({"model_type": "gradient_boosting"})
[ACT]    readmission_rate_by({"feature": "number_inpatient"})
[OBSERVE] {"encounter_id": 196770162, "risk_score": 0.561, "risk_percentile": 100.0, "baseline_readmission_rate": 0.09, "record": {"age_years": 25, "time_in_hospital": 2, "num_lab_procedures": 58, "num_procedures": 0, "num_medications": 11, "number_outpatient"...
[OBSERVE] {"model": "gradient_boosting", "AUC": 0.63, "readmissions_caught_in_top_20pct": 0.355, "readmit_rate_in_top_20pct": 0.159, "overall_readmit_rate": 0.09, "note": "AUC 0.5 = random, 1.0 = perfect. Published models on this data reach ~0.65-0.70."}
[OBSERVE] {"feature": "number_inpatient", "overall_rate": 0.0898, "groups": [{"group": "0", "patients": 61792, "readmission_rate": 0.0813}, {"group": "1", "patients": 5798, "readmission_rate": 0.1288}, {"group": "2+", "patients": 2397, "readmission_rate": 0.21...

--- Step 3 ---
[THINK]  I notice the untested A1C, decreased insulin dose, and young age could hint at possible type 1 diabetes with recurrent DKA, though I shouldn't claim a diagnosis outright—better to frame it as something worth flagging for assessment. I should also note the dataset shows 8 prior inpatient visits in the past year, and keep the whole response under 300 words.

====================================================================================================
FINAL ANSWER  (3 steps, 5 tool calls, API cost ≈ $0.050)
====================================================================================================

Highest-Risk Patients and Discharge Briefing

The 5 highest-risk patients (historical test set)

Encounter	Predicted risk	Age	Primary diagnosis	Inpatient stays in prior year	Discharged to
196770162	56%	25	Diabetes	8	Home
160455492	51%	75	Other	2	Rehab/inpatient
35170302	51%	35	Injury	11	Home
64059204	50%	65	Diabetes	1	Rehab/inpatient
72530760	49%	75	Diabetes	1	Rehab/inpatient

The average readmission rate is 9%, so these risks are about 5–6 times higher. These are past records: 3 of the 5 were actually readmitted within 30 days.

Briefing: Encounter 196770162 (25-year-old woman, 56% predicted risk, the highest in the test set)

Why she is high risk: - 8 hospital stays in the past year. This is by far the biggest factor in her score. Across the dataset, patients with 2 or more prior stays were readmitted 21% of the time, compared with 8% for those with none. - She came in through the emergency room with diabetes as the main diagnosis, and had 1 prior ER visit. - Her insulin was lowered during this stay. - Gaps in her record: her A1C and blood glucose were not tested. - Her young age, short 2-day stay and discharge home lowered her score slightly.

Caveats: The model is only moderately accurate (AUC 0.63, where 0.5 is a coin flip and 1.0 is perfect). It shows which patients tend to be readmitted, not why. The record doesn’t say why she keeps being admitted. Her pattern may suggest unstable diabetes, possibly type 1 with repeated DKA, but that is a guess for the team to confirm.

Interventions to consider

Before discharge - Check her A1C and look into why she has been admitted so often, including DKA, access to insulin and supplies, cost, mental health and eating disorders. - Check that the lower insulin dose is right and that she can manage it herself. Use teach-back on insulin, sick-day rules and when to seek care. - Refer her to diabetes education and endocrinology. Do a social-needs screen.

After discharge - Follow-up visit within 7 days, and a phone call within 48–72 hours. - Assign a case manager or care-transition program because of her repeated stays. - Clear contact for glucose problems, so she has an option besides the ER.

For the program: Flag every patient with 2 or more stays in the past year for this kind of planning. The care team makes the final clinical decisions.

--- Step 2 ---
[ACT]    get_patient_profile({"encounter_id": 190450968})
[ACT]    evaluate_model({"model_type": "gradient_boosting"})
[ACT]    readmission_rate_by({"feature": "discharge_to"})
[ACT]    readmission_rate_by({"feature": "number_inpatient"})
[OBSERVE] {"encounter_id": 190450968, "risk_score": 0.564, "risk_percentile": 100.0, "baseline_readmission_rate": 0.09, "record": {"age_years": 85, "time_in_hospital": 9, "num_lab_procedures": 27, "num_procedures": 3, "num_medications": 18, "number_outpatient"...
[OBSERVE] {"model": "gradient_boosting", "AUC": 0.654, "readmissions_caught_in_top_20pct": 0.394, "readmit_rate_in_top_20pct": 0.177, "overall_readmit_rate": 0.09, "note": "AUC 0.5 = random, 1.0 = perfect. Published models on this data reach ~0.65-0.70."}
[OBSERVE] {"feature": "discharge_to", "overall_rate": 0.0898, "groups": [{"group": "Home", "patients": 44320, "readmission_rate": 0.0695}, {"group": "Home with health services", "patients": 8364, "readmission_rate": 0.0954}, {"group": "Left against medical adv...
[OBSERVE] {"feature": "number_inpatient", "overall_rate": 0.0898, "groups": [{"group": "0", "patients": 61792, "readmission_rate": 0.0813}, {"group": "1", "patients": 5798, "readmission_rate": 0.1288}, {"group": "2+", "patients": 2397, "readmission_rate": 0.21...



--- Step 3 ---
[THINK]  I should flag that A1C and glucose weren't tested, which is a clinical gap rather than a model driver, and keep the whole writeup under 300 words. I also want to note that discharge to another inpatient facility could reflect transfers being miscounted as readmissions—a known dataset caveat—and that the model's AUC of 0.654 suggests only moderate accuracy. Worth mentioning patient 2's eight prior admissions too.

====================================================================================================
FINAL ANSWER  (3 steps, 6 tool calls, API cost ≈ $0.053)
====================================================================================================

Top 5 highest-risk patients (test set)

Encounter	Predicted risk	Age	Primary dx	Prior admissions	Discharged to
190450968	56.4%	85	Other	3	Other inpatient/rehab
144644334	51.7%	55	Circulatory	8	Home
76366848	50.5%	75	Injury	2	Other inpatient/rehab
72530760	49.6%	75	Diabetes	1	Other inpatient/rehab
170502462	47.5%	85	Other	1	Other inpatient/rehab

The average readmission rate is 9%. These patients’ predicted risks are about 5–6 times higher.

Briefing: Patient 190450968 (85-year-old man, highest risk)

Why the model flags him (largest factors first): - 3 inpatient admissions in the past year. This is the biggest factor. Patients with 2 or more prior admissions are readmitted 21.5% of the time, versus 8.1% for those with none. - Discharge to another inpatient or rehab facility. This group is readmitted 20.3% of the time, versus 7.0% for patients sent home. - Smaller factors: a 9-day stay, age 85, and 9 diagnoses. - Factors lowering his risk: he was admitted electively and arrived as a transfer.

A gap worth noting: He takes insulin (unchanged dose), but neither his A1C nor his glucose was tested during the stay.

Caveats

The model is only moderately accurate (AUC 0.654; it catches 39% of readmissions in the top-risk 20%). Use it to rank patients, not to decide care for one person.
These are associations, not causes. A rehab discharge likely reflects how sick the patient is, not harm from the facility. In this kind of data, some “readmissions” after facility discharge may actually be transfers.
The data is from 1999–2008.

Interventions to consider (care team decides)

Before discharge - Review why he was admitted 3 times: is there an unresolved condition or a pattern? - Check A1C and glucose, and review his insulin plan. Older adults are at risk of low blood sugar (hypoglycemia). - Review all 18 medications for interactions and drugs that are risky in older adults. - Hand off to the rehab facility doctor-to-doctor: a written plan for insulin and glucose, warning signs, and who to call.

After discharge - Contact the facility within 48–72 hours. - Book a primary care or diabetes follow-up within 7–14 days. - Set clear rules for when to escalate glucose readings or symptoms, so problems are handled before he needs readmission. - Patient 144644334 (8 prior admissions, going home) should also be prioritized for transitional care.

5. Traditional ML vs. AI agent

	Traditional ML model	AI agent (LLM + tools + loop)
Output	A number (risk score)	Analysis, explanation, recommendations
Who decides the analysis steps?	The data scientist writes the code	The agent plans and chooses tools
New question	Write new code	Ask in plain English
Explains itself	Coefficients / importances (needs expertise)	Plain-language reasoning tied to evidence
Speed & cost	Milliseconds, essentially free	Seconds per step, costs money per API call
Determinism	Same input -> same output	Can vary between runs
Main risk	Biased or poorly calibrated model	Above, and hallucination and over confident language

Limitations and responsible use

Hallucination risk: we reduce it by forcing the agent to use tools and telling it to never invent numbers. A human should still verify key claims.
Correlation =/= causation: e.g., prior inpatient visits predict readmission, but don’t cause it.
Data age & bias: 1999–2008 data from 130 US hospitals, which may not reflect today’s care or every population.
Privacy: in a real deployment, patient data sent to an LLM must comply with HIPAA (a business associate agreement, de-identification). Our design only sends aggregated tool outputs, never the raw dataset.
Human in the loop: the agent is decision support. Clinicians make the final decisions.
Guardrails: limited tool set (no arbitrary code execution), step limit, cost tracking, and error handling.

References

Strack, B. et al. (2014). Impact of HbA1c Measurement on Hospital Readmission Rates: Analysis of 70,000 Clinical Database Patient Records. BioMed Research International.
Yao, S. et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR.
Anthropic (2024). Building Effective Agents. https://www.anthropic.com/research/building-effective-agents
Centers for Medicare & Medicaid Services. Hospital Readmissions Reduction Program (HRRP).

AI Agents Hands-On Demo: Predicting 30-Day Hospital Readmissions for Diabetic Patients

Contents

What is an AI agent?

0. Setup

1. Data Processing

2. Traditional machine learning baseline

3. Building the AI agent

3a. Tools: the agent’s “hands”

TOOL 1: describe_dataset

TOOL 2: readmission_rate_by

TOOL 3: evaluate_model

TOOL 4: get_feature_importance

TOOL 5: list_high_risk_patients

TOOL 6: get_patient_profile

TOOL 7: plot_readmission_rate

3b. Tool definitions: how the LLM learns what each tool does

3c. The model and the agent loop

4. Agent demos

Demo 1: Open-ended investigation for the quality committee

30-Day Readmission Drivers: Diabetic Inpatients

Strongest drivers (both models agree, and the raw rates confirm it)

How well can we predict it?

Caveats

Recommendations

Diabetic 30-Day Readmissions: Key Drivers and How Well We Can Predict Them

Strongest drivers (both models agree, and the raw rates confirm it)

How reliable is prediction?

Caveats

Recommendations

Demo 2: Testing a published claim

Short answer: only a weak signal, and it may not mean much

Other factors explain most of it

Recommendations

Does testing HbA1c in hospital go with fewer readmissions?

What the chart shows

Other factors that could explain it

Caveats

Recommendations

Demo 3: Patient-level decision support (discharge planning)

Highest-Risk Patients and Discharge Briefing

The 5 highest-risk patients (historical test set)

Briefing: Encounter 196770162 (25-year-old woman, 56% predicted risk, the highest in the test set)

Interventions to consider

Top 5 highest-risk patients (test set)

Briefing: Patient 190450968 (85-year-old man, highest risk)

Caveats

Interventions to consider (care team decides)

5. Traditional ML vs. AI agent

Limitations and responsible use

References