Three hymns, sung on the beat. Assembled from sixteen takes, measured, then heard.
Amazing Grace, America the Beautiful and the Battle Hymn of the Republic, every verse, sung by a synthetic voice over piano arrangements written for each hymn: each phrase chosen by a listener, placed on the score by warping, checked by two timing instruments and a pitch tracker, and passed by ear. Part of AI Jam Sessions, an MCP server that teaches AI to read, play and sing music.
Install
npm install -g @mcptoolshop/ai-jam-sessions
Play
ai-jam-sessions play fur-elise --engine piano --speed 0.7
Sing
ai-jam-sessions sing america-the-beautiful --with-piano
Listen: three hymns, sung on the beat
Amazing Grace, America the Beautiful and the Battle Hymn of the Republic, every verse, sung by a synthetic voice over piano arrangements written for each hymn. Press play and the score follows the voice.
Music: Traditional (New Britain, American, 1829). Words: John Newton (verses 1-3, 1779); verse 4 from A Collection of Sacred Ballads (1790).
Music: Samuel A. Ward. Words: Katharine Lee Bates.
Music: Anonymous (the tune "Glory, Hallelujah", in print by 1859; often credited to William Steffe). Words: Julia Ward Howe (The Atlantic Monthly, February 1862).
This browser could not start WebGL, so the 3D view is off. The recordings above still play.
Press play on either hymn: the view follows the voice, and the words light as they are sung.
What the gates measured on these performances
| Amazing Grace | America the Beautiful | The Battle Hymn of the Republic | |
|---|---|---|---|
| Syllables whose vowel lands within 40 ms of the score | 107 of 112 | 217 of 224 | 285 of 414 |
| Notes within 50 cents of the score | 135 of 140 | 218 of 224 | 368 of 424 |
| Words a local listener heard, per take | 97–99% | 87–95% | 92–99% |
| Built from | 16 phrases from 11 takes | 20 phrases from 11 takes | 40 phrases from 16 takes |
| The Director's ear | Pass | Pass | Pass |
Timing is checked by two instruments: an energy detector for the vowel's onset, cross-checked by a singing forced aligner, and a syllable fails only when both say it is off. Pitch is tracked per note with two trackers, and a note counts as off only when both say so. The Battle Hymn's quick dotted notes are left on the singer's own timing, the way a singer smooths them, so most of its timing misses are by design. These are honest numbers, not a pass/fail stamp: the misses are listed in the receipts.
The voice is synthetic: SoulX-Singer (Apache-2.0), sung from the score, with the timbre of SoulX-Singer's own example voice. It imitates no real singer. The piano arrangements were written for this project by Kimi-K3 (Moonshot AI) and are dedicated to the public domain (CC0), played on this project's own oscillator grand. All three hymns are in the public domain.
How a take becomes a performance
No single render was good enough on its own. Each performance is assembled from sixteen, and measured before it is called a mix.
A score clock
Every syllable gets its pitch, its onset and its length from the hymn's own arrangement, on the same timeline the piano plays. Held notes, rests and breaths included.
A sound check first
Before a song is rendered, the same voice sings a sixteen-word calibration phrase at the song's tempo. Its timing footprint, per consonant group and for held against short notes, says how many takes the song needs and which words are risky.
Sixteen takes on a rented GPU
A score-conditioned singer renders each hymn sixteen times on a rented A40 through offrig, so the local card stays free. Each phrase is sung with a second of silence in front, then trimmed, which keeps the model's start-up noise out.
One take per phrase
A local listener transcribes every phrase of every take without being given the lyrics, and pitch is measured note by note. Each phrase comes whole from the take that was understood best and sung most in tune.
Placed by warping, not cutting
Each phrase is time-warped onto the score so every vowel lands on its beat, with nothing replayed or skipped inside the phrase. Takes meet only at the rests between phrases.
Two gates, two instruments
Timing: an energy detector dates each vowel, a singing forced aligner checks it, and a syllable fails only when both say it is off. Pitch: every note is tracked against the score in cents.
A person listens last
The listening review: press M wherever something sounds wrong, and a report joins each mark to the take, the join and the gates at that moment. A listener's ear settles where something sounds off; naming the cause weighs more from training. Its first marks found two defects no gate could see, both fixed at the root. It is moving into the cockpit next.
The full route, its receipts and what is still imperfect: handbook → Vocals.
What makes it tick
Five senses for a model that has none.
Reading
Real MIDI sheet music with deep musical annotations — parsed, analyzed, and explained. Not hand-written approximations.
Hearing
Six audio engines play through your speakers so the humans in the room become the AI's ears.
Seeing
Piano roll renders as SVG the model can read back. Interactive guitar tablature. Browser cockpit with visual keyboard.
Remembering
Practice journal persists across sessions. Next time, the AI reads its journal and picks up where it left off.
Singing
Whole hymns sung by a score-conditioned singer and placed on the piano's clock (listen above), plus vocal tract synthesis with 20 voice presets and a sing-along mode with solfege, contour and syllable narration.
56 MCP Tools + 4 Prompts
Learn, play, sing, build, score — browse songs, transpose keys, mute hands, preview teaching cues, render piano rolls, write annotations, and journal reflections.
A Training Dataset to Match
jam-actions-v0 — a public corpus of 57 multi-turn MCP tool-use traces over classical piano, with a 7-axis release gate and cold-start reproducibility. CC-BY-SA-3.0-DE.
jam-actions-v0 — a public training dataset
Multi-turn MCP tool-use traces over real classical piano. Grounded tool-use over symbolic music — not just text generation.
57 records · 4 piano pieces
Public subset 0.6.0: four piano-midi.de arrangements (Bach, Beethoven, Debussy, Mozart), 45 train and 12 held out. Version 0.6.0 withdrew 58 records from four songs whose files had no established arrangement licence. The 16-record canonical baseline was measured on the earlier composition.
7-axis release gate
Absolute floor, margin compound, tool-use rate, correct-after-tool, misinterpretation count, stratum floor (all blocking); enriched-vs-non reporting (informational). Admits a ceiling-saturated bucket so trivial wins do not dilute harder strata.
Reproducible in under a minute
pnpm install, run the checksum verifier, run the release-gate CLI against the canonical baseline. .gitattributes pins LF for sha256 + dataset tree so it works on Windows native, macOS, Linux, and WSL.
Slice 22 baseline PASSES
The revised gate (axes 2 + 6 with ceiling_saturated_pass) admits the canonical baseline cleanly. The Slice 19 baseline still FAILS — kept as a regression diagnostic so the gate has teeth.
Cited and licensed end-to-end
CITATION.cff, Zenodo deposition metadata (13 fields, ISO 639-3 language code), CC-BY-SA-3.0-DE preserved across the MIDI arrangements (Krueger) and the annotations + traces + evals (mcp-tool-shop-org).
Honest about provenance
Version 0.6.0 withdraws Chopin's Nocturne Op. 9 No. 2 and Prelude Op. 28 No. 4, Beethoven's Pathétique II, and Schumann's Träumerei. Those files came from midiworld.com and bitmidi.com and carry no established arrangement licence. The remaining 57 records are byte-identical to 0.5.0.
Fine-tuning receipts, three arcs
Preregistered fine-tunes on sealed baselines: v0 was an honest negative (0.661 → 0.601); v1 hit +0.202 but missed its frozen 13/16 bar by one; B-1 cleared its ex-ante bar (0.678 → 0.890, 29/36 wins). The five seed adapters are withdrawn: they were trained on records that include the four songs 0.6.0 removed. The measurements stand as recorded.
Use the dataset
Verify the package
git clone https://github.com/mcp-tool-shop-org/ai-jam-sessions.git
cd ai-jam-sessions && pnpm install
# 274 checksum entries; ~2 seconds.
pnpm exec tsx scripts/verify-public-package-checksums.tsReproduce the canonical PASS
pnpm exec tsx scripts/check-release-gate.ts \
datasets/jam-actions-v0-public/evals/\
slice21-fair-e3-baseline-results.json
# → "Aggregate: PASS" (exit 0)Read the dataset card
# Full HF-format dataset card + YAML frontmatter
open datasets/jam-actions-v0-public/README.md
# Zenodo deposition metadata (13 fields)
cat datasets/jam-actions-v0-public/zenodo-metadata.jsonCite it
# Citation File Format (CFF)
cat datasets/jam-actions-v0-public/CITATION.cff
# Bernd Krueger (piano-midi.de) for the MIDI arrangements.
# mcp-tool-shop-org for the annotations, traces, and evals.
# Both under CC-BY-SA-3.0-DE.Quick start
Install globally
npm install -g @mcptoolshop/ai-jam-sessionsClaude Desktop config
{
"mcpServers": {
"ai_jam_sessions": {
"command": "npx",
"args": [
"-y", "-p",
"@mcptoolshop/ai-jam-sessions",
"ai-jam-sessions-mcp"
]
}
}
}Play a song
# Play Fur Elise at 70% speed
ai-jam-sessions play fur-elise --speed 0.7
# View piano roll as SVG
ai-jam-sessions view autumn-leaves --measures 1-16
# Sing along with piano
ai-jam-sessions sing imagine --with-pianoGuitar tablature
# View interactive guitar tab
ai-jam-sessions view-guitar greensleeves
# List tunings and voices
ai-jam-sessions list --genre folkSound engines
Six engines plus a layered combinator that runs any two simultaneously.
| Engine | Type | Character |
|---|---|---|
| Oscillator Piano | Additive synthesis | Multi-harmonic piano with hammer noise, 48-voice polyphony, stereo imaging |
| Sample Piano | WAV playback | Salamander Grand Piano — 480 samples, 16 velocity layers, 88 keys (programmatic API; samples user-supplied) |
| Vocal (Sample) | Pitch-shifted samples | Sustained vowel tones with portamento and legato mode |
| Vocal Tract | Physical model | LF glottal waveform through 44-cell digital waveguide. Soprano, alto, tenor, bass |
| Vocal Synth | Additive synthesis | 15 Kokoro voice presets with formant shaping, breathiness, vibrato |
| Guitar | Additive synthesis | Physically-modeled plucked string — 4 presets, 8 tunings, 17 tunable parameters |
| Layered | Combinator | Wraps two engines, dispatches every MIDI event to both |
Song library
109 annotated songs across 12 genres. Each genre has a deeply annotated exemplar — now including a public-domain patriotic hymn.
| Genre | Exemplar | Key | Teaches |
|---|---|---|---|
| Blues | The Thrill Is Gone | B minor | Minor blues form, call-and-response |
| Classical | Fur Elise | A minor | Rondo form, touch differentiation |
| Film | Comptine d'un autre ete | E minor | Arpeggiated textures, dynamic architecture |
| Folk | America the Beautiful | F major | Tonic-dominant hymn harmony, patriotic phrasing, model-friendly range |
| Jazz | Autumn Leaves | G minor | ii-V-I progressions, swing eighths |
| Latin | The Girl from Ipanema | F major | Bossa nova rhythm, chromatic modulation |
| New-Age | River Flows in You | A major | I-V-vi-IV recognition, flowing arpeggios |
| Pop | Imagine | C major | Arpeggiated accompaniment, restraint |
| Ragtime | The Entertainer | C major | Oom-pah bass, syncopation |
| R&B | Superstition | Eb minor | 16th-note funk, percussive keyboard |
| Rock | Your Song | Eb major | Piano ballad voice-leading |
| Soul | Lean on Me | C major | Diatonic melody, gospel accompaniment |
MCP tools
56 tools and 4 prompt templates across eight categories: Learn, Play, Practice, Sing, Guitar, Build, Score, and Listen.
| Tool | Category | Description |
|---|---|---|
| `list_songs` | Learn | Browse by genre, difficulty, or keyword |
| `song_info` | Learn | Full musical analysis — structure, key moments, teaching goals |
| `suggest_song` | Learn | Recommendation based on genre, difficulty, and play history |
| `practice_setup` | Learn | Recommended speed, mode, voice settings for a song |
| `compare_songs` | Learn | Cross-genre pattern recognition — key relationships, shared forms, teaching connections |
| `play_song` | Play | Play through speakers — any engine, speed, mode, measure range |
| `view_piano_roll` | Play | Render as SVG (hand color or pitch-class chromatic rainbow) |
| `sing_along` | Sing | Singable text — note-names, solfege, contour, or syllables |
| `view_guitar_tab` | Guitar | Render interactive guitar tablature as HTML |
| `tune_guitar` | Guitar | Adjust any parameter of any guitar voice |
| `annotate_song` | Build | Write musical language for a raw song and promote it |
| `save_practice_note` | Build | Journal entry with auto-captured session data |
| `tune_keyboard` | Build | Adjust any parameter of any keyboard voice |
| `score_performance` | Score | Score a MIDI play-along — pitch accuracy, timing, graded feedback |
| `score_annotation` | Score | Score annotation quality across 5 dimensions |
| `annotation_progress` | Score | Track annotation quality across the library |