Why the test is blind
We hide the model names because knowing them changes what you hear. A well-known name or a higher price pushes a score before the first note. In a blind test the only input is the sound.
Labels matter most when ears cannot settle a question. In a Deezer and Ipsos survey of 9,000 people, 97% could not tell fully AI-generated music from human-made music (Deezer and Ipsos, 12 Nov 2025). So our scoring page shows the clips as A, B and C, in an order that is shuffled per brief. Model averages are revealed only after every clip has a score.
The seven briefs
The seven briefs run from easy to hard. Six have Turkish lyrics and one has no vocal. A brief is the full request a model receives: a genre description, a voice and the lyrics. Every model got the same brief and produced one clip of about 90 seconds.
| Brief | What it measures |
|---|---|
| 1. Afro house vocal chops | The easiest case: short, repeated phrases whose words do not have to be understood |
| 2. Deep house soul hook | Short lines that must be sung clearly |
| 3. Afro house verse and hook | Whether words sit on the rhythm and the verse leads into the hook |
| 4. Melodic house with a male vocal | Turkish on a male voice |
| 5. Long Turkish verse | The hard case: eight lines packed with Turkish-specific sounds |
| 6. Afro house instrumental | A control with no vocal, to judge production alone |
| 7. Turkish pop | A control outside house music |
The lyrics include the sounds we expected models to struggle with: the soft g (ğ) in yağmur and düğün, the dotless i (ı) in ışık and kırık, ö and ü in gözlerin and gülüm, and ş and ç in şehir and çiçek.
The three scores
Each clip gets three scores from 1 to 5. Each score answers one plain question.
- Pronunciation. Can a Turkish speaker understand the words, and are they the right words?
- Production. Do the mix and the sound fit the genre, and could you play it in a club?
- Overall. Would you pay for this result?
Pronunciation is averaged over the six briefs that have a vocal, because the instrumental has no words to score. Production and overall are averaged over all seven.
Results of the 21 Sep 2026 test
The model behind Lyro High led on all three scores. The second model was weak on Turkish pronunciation but won the vocal-chop brief. The third came last for production and overall. We describe the models by what they are to Lyro today, not by their makers’ names.
| Model | Pronunciation | Production | Overall | What we did with it |
|---|---|---|---|---|
| The model behind Lyro High | 4.67 | 4.43 | 4.57 | Lyro’s default for sung lyrics |
| A second model we no longer offer | 2.00 | 3.00 | 3.29 | Offered for vocal chops and tempo-locked beats until 24 Sep 2026 |
| A third model we tested and did not offer | 2.17 | 2.71 | 2.86 | Not offered |
On the vocal-chop brief the second model scored 5 overall against 4 for the model behind Lyro High, even though its pronunciation on that brief scored 1. When words are chopped into rhythm, clarity matters less than groove. The model behind Lyro High had the highest overall score on the other six briefs, level with the third model on one of them.
Lyro Extra was added on 24 Sep 2026, after this test, so it is not in the table. We checked it for Turkish separately: one Turkish song on each model, transcribed with speech-to-text software. Lyro High came back word for word; Lyro Extra lost a verse. That is one song, not a blind test, but it is why the app holds Lyro Extra back for Turkish lyrics.
The limits of this test
This was a small internal test, and it should be read as one. We publish it because it drives our routing, not because it settles which model is best.
- One rater. One Turkish speaker on our team scored every clip.
- One language. All lyrics were Turkish. The scores say nothing about the other languages Lyro supports.
- Small sample. There were 21 clips: one generation per model per brief, from a single seed, the number that fixes a model’s random starting point. A second run could move any score.
- Narrow genres. Six of the seven briefs were house music.
- Three models. We tested models available to us through fal.ai. Suno and Udio were not part of the test, so these scores say nothing about them. Lyro Extra came later and is not part of it either.
- One date. The results describe the model versions available on 21 Sep 2026.
How the results become routing rules
Routing means choosing which model receives your request. Lyro offers two models today, Lyro High and Lyro Extra, and its rules follow one question: how well do the words need to be understood, and in which language?
- Lyro High is the default. The model behind it won the test, so sung lyrics and instrumentals go to Lyro High unless you pick otherwise. A song costs 4 credits, whatever its length.
- Sung Turkish goes to Lyro High. Lyro Extra is held back for Turkish lyrics unless you confirm that you want it anyway. Lyro detects Turkish even when it is typed without Turkish letters.
- Lyro Extra is the second choice. By its provider’s account it gives richer arrangements and more expressive vocals, and it is made for English, German, Spanish, French, Hindi, Japanese, Korean and Portuguese. A song costs 5 credits. We have not blind-tested it.
- Vocal chops are made on Lyro High or Lyro Extra. The model that won the vocal-chop brief is no longer offered, so this is the one place where the test points one way and Lyro’s line-up another.
- Work on your own audio runs in Lyro’s studio tools. A beat written around your vocal, or a vocal improvised over your beat, follows the tempo and key of your file.
These rules come from one test in one language and one later check. We have not run a test for each genre or each language, so outside sung Turkish they are a working rule, not a measurement. Lyro recommends a model, and you can choose another.
How the test continues in the app
Every generated track in the app has a thumbs up and a thumbs down, and those votes continue the test at a scale one rater cannot reach. We group them by model, genre and language. If real use disagrees with the blind test, we test again.
Votes are not blind, so we treat them as a signal to re-test, not as a final score. A new model is checked before it receives requests; Lyro Extra had the Turkish check described above, and it will go through the next blind test. You can hear the current routing in the samples on the homepage.
Questions
Which AI music model is best for Turkish vocals?
In our small internal blind test on 21 Sep 2026, the model behind Lyro High scored 4.67 out of 5 for Turkish pronunciation; the two other models scored 2.00 and 2.17. It was a small test with one rater. More in our Turkish guide.
Why did a lower-scoring model win one brief?
The brief was vocal chops, where words are cut into short rhythmic phrases and do not have to be understood. A second model scored 5 overall there against 4 for the model behind Lyro High. Lyro offered that model for vocal chops until 24 Sep 2026; it no longer does, and chops are now made on Lyro High or Lyro Extra.
Did you test Suno or Udio?
No. The test covered three models available through fal.ai, one of which is the model behind Lyro High. The scores say nothing about Suno or Udio. Our comparison pages stick to dated, sourced facts.
How many people scored the clips?
One. A Turkish speaker on our team scored all 21 clips without seeing the model names. That is a real limit, which is why votes in the app are used to check the results.