Skip to content
♪AI Jam Sessions
Listen first

Three hymns, sung on the beat. Assembled from sixteen takes, measured, then heard.

Amazing Grace, America the Beautiful and the Battle Hymn of the Republic, every verse, sung by a synthetic voice over piano arrangements written for each hymn: each phrase chosen by a listener, placed on the score by warping, checked by two timing instruments and a pitch tracker, and passed by ear. Part of AI Jam Sessions, an MCP server that teaches AI to read, play and sing music.

Install

npm install -g @mcptoolshop/ai-jam-sessions

Play

ai-jam-sessions play fur-elise --engine piano --speed 0.7

Sing

ai-jam-sessions sing america-the-beautiful --with-piano

Listen: three hymns, sung on the beat

Amazing Grace, America the Beautiful and the Battle Hymn of the Republic, every verse, sung by a synthetic voice over piano arrangements written for each hymn. Press play and the score follows the voice.

Amazing GraceNew Britain · G major · 72 BPM · 4 verses · 2:48

Music: Traditional (New Britain, American, 1829). Words: John Newton (verses 1-3, 1779); verse 4 from A Collection of Sacred Ballads (1790).

America the BeautifulMaterna · Bb major · 92 BPM · 4 verses · 3:21

Music: Samuel A. Ward. Words: Katharine Lee Bates.

The Battle Hymn of the RepublicBattle Hymn · G major · 76 BPM · 5 verses · 4:27

Music: Anonymous (the tune "Glory, Hallelujah", in print by 1859; often credited to William Steffe). Words: Julia Ward Howe (The Atlantic Monthly, February 1862).

This browser could not start WebGL, so the 3D view is off. The recordings above still play.

Press play on either hymn: the view follows the voice, and the words light as they are sung.

What the gates measured on these performances

Amazing GraceAmerica the BeautifulThe Battle Hymn of the Republic
Syllables whose vowel lands within 40 ms of the score107 of 112217 of 224285 of 414
Notes within 50 cents of the score135 of 140218 of 224368 of 424
Words a local listener heard, per take97–99%87–95%92–99%
Built from16 phrases from 11 takes20 phrases from 11 takes40 phrases from 16 takes
The Director's earPassPassPass

Timing is checked by two instruments: an energy detector for the vowel's onset, cross-checked by a singing forced aligner, and a syllable fails only when both say it is off. Pitch is tracked per note with two trackers, and a note counts as off only when both say so. The Battle Hymn's quick dotted notes are left on the singer's own timing, the way a singer smooths them, so most of its timing misses are by design. These are honest numbers, not a pass/fail stamp: the misses are listed in the receipts.

The voice is synthetic: SoulX-Singer (Apache-2.0), sung from the score, with the timbre of SoulX-Singer's own example voice. It imitates no real singer. The piano arrangements were written for this project by Kimi-K3 (Moonshot AI) and are dedicated to the public domain (CC0), played on this project's own oscillator grand. All three hymns are in the public domain.

How a take becomes a performance

No single render was good enough on its own. Each performance is assembled from sixteen, and measured before it is called a mix.

A score clock

Every syllable gets its pitch, its onset and its length from the hymn's own arrangement, on the same timeline the piano plays. Held notes, rests and breaths included.

A sound check first

Before a song is rendered, the same voice sings a sixteen-word calibration phrase at the song's tempo. Its timing footprint, per consonant group and for held against short notes, says how many takes the song needs and which words are risky.

Sixteen takes on a rented GPU

A score-conditioned singer renders each hymn sixteen times on a rented A40 through offrig, so the local card stays free. Each phrase is sung with a second of silence in front, then trimmed, which keeps the model's start-up noise out.

One take per phrase

A local listener transcribes every phrase of every take without being given the lyrics, and pitch is measured note by note. Each phrase comes whole from the take that was understood best and sung most in tune.

Placed by warping, not cutting

Each phrase is time-warped onto the score so every vowel lands on its beat, with nothing replayed or skipped inside the phrase. Takes meet only at the rests between phrases.

Two gates, two instruments

Timing: an energy detector dates each vowel, a singing forced aligner checks it, and a syllable fails only when both say it is off. Pitch: every note is tracked against the score in cents.

A person listens last

The listening review: press M wherever something sounds wrong, and a report joins each mark to the take, the join and the gates at that moment. A listener's ear settles where something sounds off; naming the cause weighs more from training. Its first marks found two defects no gate could see, both fixed at the root. It is moving into the cockpit next.

The full route, its receipts and what is still imperfect: handbook → Vocals.

What makes it tick

Five senses for a model that has none.

Reading

Real MIDI sheet music with deep musical annotations — parsed, analyzed, and explained. Not hand-written approximations.

Hearing

Six audio engines play through your speakers so the humans in the room become the AI's ears.

Seeing

Piano roll renders as SVG the model can read back. Interactive guitar tablature. Browser cockpit with visual keyboard.

Remembering

Practice journal persists across sessions. Next time, the AI reads its journal and picks up where it left off.

Singing

Whole hymns sung by a score-conditioned singer and placed on the piano's clock (listen above), plus vocal tract synthesis with 20 voice presets and a sing-along mode with solfege, contour and syllable narration.

56 MCP Tools + 4 Prompts

Learn, play, sing, build, score — browse songs, transpose keys, mute hands, preview teaching cues, render piano rolls, write annotations, and journal reflections.

A Training Dataset to Match

jam-actions-v0 — a public corpus of 57 multi-turn MCP tool-use traces over classical piano, with a 7-axis release gate and cold-start reproducibility. CC-BY-SA-3.0-DE.

jam-actions-v0 — a public training dataset

Multi-turn MCP tool-use traces over real classical piano. Grounded tool-use over symbolic music — not just text generation.

57 records · 4 piano pieces

Public subset 0.6.0: four piano-midi.de arrangements (Bach, Beethoven, Debussy, Mozart), 45 train and 12 held out. Version 0.6.0 withdrew 58 records from four songs whose files had no established arrangement licence. The 16-record canonical baseline was measured on the earlier composition.

7-axis release gate

Absolute floor, margin compound, tool-use rate, correct-after-tool, misinterpretation count, stratum floor (all blocking); enriched-vs-non reporting (informational). Admits a ceiling-saturated bucket so trivial wins do not dilute harder strata.

Reproducible in under a minute

pnpm install, run the checksum verifier, run the release-gate CLI against the canonical baseline. .gitattributes pins LF for sha256 + dataset tree so it works on Windows native, macOS, Linux, and WSL.

Slice 22 baseline PASSES

The revised gate (axes 2 + 6 with ceiling_saturated_pass) admits the canonical baseline cleanly. The Slice 19 baseline still FAILS — kept as a regression diagnostic so the gate has teeth.

Cited and licensed end-to-end

CITATION.cff, Zenodo deposition metadata (13 fields, ISO 639-3 language code), CC-BY-SA-3.0-DE preserved across the MIDI arrangements (Krueger) and the annotations + traces + evals (mcp-tool-shop-org).

Honest about provenance

Version 0.6.0 withdraws Chopin's Nocturne Op. 9 No. 2 and Prelude Op. 28 No. 4, Beethoven's Pathétique II, and Schumann's Träumerei. Those files came from midiworld.com and bitmidi.com and carry no established arrangement licence. The remaining 57 records are byte-identical to 0.5.0.

Fine-tuning receipts, three arcs

Preregistered fine-tunes on sealed baselines: v0 was an honest negative (0.661 → 0.601); v1 hit +0.202 but missed its frozen 13/16 bar by one; B-1 cleared its ex-ante bar (0.678 → 0.890, 29/36 wins). The five seed adapters are withdrawn: they were trained on records that include the four songs 0.6.0 removed. The measurements stand as recorded.

Use the dataset

Verify the package

git clone https://github.com/mcp-tool-shop-org/ai-jam-sessions.git
cd ai-jam-sessions && pnpm install

# 274 checksum entries; ~2 seconds.
pnpm exec tsx scripts/verify-public-package-checksums.ts

Reproduce the canonical PASS

pnpm exec tsx scripts/check-release-gate.ts \
  datasets/jam-actions-v0-public/evals/\
slice21-fair-e3-baseline-results.json
# → "Aggregate: PASS" (exit 0)

Read the dataset card

# Full HF-format dataset card + YAML frontmatter
open datasets/jam-actions-v0-public/README.md

# Zenodo deposition metadata (13 fields)
cat datasets/jam-actions-v0-public/zenodo-metadata.json

Cite it

# Citation File Format (CFF)
cat datasets/jam-actions-v0-public/CITATION.cff

# Bernd Krueger (piano-midi.de) for the MIDI arrangements.
# mcp-tool-shop-org for the annotations, traces, and evals.
# Both under CC-BY-SA-3.0-DE.

Quick start

Install globally

npm install -g @mcptoolshop/ai-jam-sessions

Claude Desktop config

{
  "mcpServers": {
    "ai_jam_sessions": {
      "command": "npx",
      "args": [
        "-y", "-p",
        "@mcptoolshop/ai-jam-sessions",
        "ai-jam-sessions-mcp"
      ]
    }
  }
}

Play a song

# Play Fur Elise at 70% speed
ai-jam-sessions play fur-elise --speed 0.7

# View piano roll as SVG
ai-jam-sessions view autumn-leaves --measures 1-16

# Sing along with piano
ai-jam-sessions sing imagine --with-piano

Guitar tablature

# View interactive guitar tab
ai-jam-sessions view-guitar greensleeves

# List tunings and voices
ai-jam-sessions list --genre folk

Sound engines

Six engines plus a layered combinator that runs any two simultaneously.

EngineTypeCharacter
Oscillator PianoAdditive synthesisMulti-harmonic piano with hammer noise, 48-voice polyphony, stereo imaging
Sample PianoWAV playbackSalamander Grand Piano — 480 samples, 16 velocity layers, 88 keys (programmatic API; samples user-supplied)
Vocal (Sample)Pitch-shifted samplesSustained vowel tones with portamento and legato mode
Vocal TractPhysical modelLF glottal waveform through 44-cell digital waveguide. Soprano, alto, tenor, bass
Vocal SynthAdditive synthesis15 Kokoro voice presets with formant shaping, breathiness, vibrato
GuitarAdditive synthesisPhysically-modeled plucked string — 4 presets, 8 tunings, 17 tunable parameters
LayeredCombinatorWraps two engines, dispatches every MIDI event to both

Song library

109 annotated songs across 12 genres. Each genre has a deeply annotated exemplar — now including a public-domain patriotic hymn.

GenreExemplarKeyTeaches
BluesThe Thrill Is GoneB minorMinor blues form, call-and-response
ClassicalFur EliseA minorRondo form, touch differentiation
FilmComptine d'un autre eteE minorArpeggiated textures, dynamic architecture
FolkAmerica the BeautifulF majorTonic-dominant hymn harmony, patriotic phrasing, model-friendly range
JazzAutumn LeavesG minorii-V-I progressions, swing eighths
LatinThe Girl from IpanemaF majorBossa nova rhythm, chromatic modulation
New-AgeRiver Flows in YouA majorI-V-vi-IV recognition, flowing arpeggios
PopImagineC majorArpeggiated accompaniment, restraint
RagtimeThe EntertainerC majorOom-pah bass, syncopation
R&BSuperstitionEb minor16th-note funk, percussive keyboard
RockYour SongEb majorPiano ballad voice-leading
SoulLean on MeC majorDiatonic melody, gospel accompaniment

MCP tools

56 tools and 4 prompt templates across eight categories: Learn, Play, Practice, Sing, Guitar, Build, Score, and Listen.

ToolCategoryDescription
`list_songs`LearnBrowse by genre, difficulty, or keyword
`song_info`LearnFull musical analysis — structure, key moments, teaching goals
`suggest_song`LearnRecommendation based on genre, difficulty, and play history
`practice_setup`LearnRecommended speed, mode, voice settings for a song
`compare_songs`LearnCross-genre pattern recognition — key relationships, shared forms, teaching connections
`play_song`PlayPlay through speakers — any engine, speed, mode, measure range
`view_piano_roll`PlayRender as SVG (hand color or pitch-class chromatic rainbow)
`sing_along`SingSingable text — note-names, solfege, contour, or syllables
`view_guitar_tab`GuitarRender interactive guitar tablature as HTML
`tune_guitar`GuitarAdjust any parameter of any guitar voice
`annotate_song`BuildWrite musical language for a raw song and promote it
`save_practice_note`BuildJournal entry with auto-captured session data
`tune_keyboard`BuildAdjust any parameter of any keyboard voice
`score_performance`ScoreScore a MIDI play-along — pitch accuracy, timing, graded feedback
`score_annotation`ScoreScore annotation quality across 5 dimensions
`annotation_progress`ScoreTrack annotation quality across the library