<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet href="/scripts/pretty-feed-v3.xsl" type="text/xsl"?><rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:h="http://www.w3.org/TR/html4/"><channel><title>osmandagdeviren.com.tr</title><description>Stay hungry, stay foolish</description><link>https://osmandagdeviren.com.tr</link><language>en</language><item><title>14 LLMs Benchmarked on Turkish &amp; English NER 🔬</title><link>https://osmandagdeviren.com.tr/en/blog/ner-llm-benchmark</link><guid isPermaLink="true">https://osmandagdeviren.com.tr/en/blog/ner-llm-benchmark</guid><description>Which LLM best extracts named entities from news articles? I tested 14 models in Turkish and English — real data, honest numbers, surprising winner.</description><pubDate>Wed, 10 Jun 2026 12:00:00 GMT</pubDate><content:encoded>&lt;h2&gt;Why LLMs Instead of a Dedicated NER Model? 🤷&lt;/h2&gt;
&lt;p&gt;spaCy and Stanza exist for exactly this. They&apos;re fast, cheap, and purpose-built. The problem is Turkish morphology.&lt;/p&gt;
&lt;p&gt;Turkish is agglutinative. A proper noun like &lt;em&gt;Bozbey&lt;/em&gt; becomes &lt;em&gt;Bozbey&apos;in&lt;/em&gt; (possessive), &lt;em&gt;Bozbey&apos;e&lt;/em&gt; (dative), &lt;em&gt;Ankara Adliyesi&lt;/em&gt; (organization) becomes a surface form that looks nothing like &lt;em&gt;Ankara&lt;/em&gt; alone. Institutional names get long: &quot;Bursa Cumhuriyet Başsavcılığı&quot; contains &quot;Bursa&quot; but they are not the same entity — extracting both would be wrong. Suffix-heavy morphology historically beats up off-the-shelf models trained mostly on English.&lt;/p&gt;
&lt;p&gt;So I tested LLM extraction through OpenRouter instead: one API, one prompt, both languages.&lt;/p&gt;
&lt;p&gt;The question: &lt;strong&gt;which model gives the best named-entity F1 per dollar — especially on Turkish?&lt;/strong&gt;&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The Candidates 🏁&lt;/h2&gt;
&lt;p&gt;14 models from the current OpenRouter catalog, prices from 2026-06-10:&lt;/p&gt;
&lt;p&gt;| Model | Provider | $/M in | $/M out |
|---|---|---|---|
| Gemini 2.5 Flash | Google | $0.30 | $2.50 |
| DeepSeek V4 Flash | DeepSeek | $0.0983 | $0.1966 |
| DeepSeek V4 Pro | DeepSeek | $0.435 | $0.87 |
| Tencent Hy3 Preview | Tencent | $0.063 | $0.21 |
| MiniMax M3 | MiniMax | $0.30 | $1.20 |
| Xiaomi MiMo-V2.5 | Xiaomi | $0.14 | $0.28 |
| Xiaomi MiMo-V2.5-Pro | Xiaomi | $0.435 | $0.87 |
| Gemini 3.5 Flash | Google | $1.50 | $9.00 |
| Gemini 3.1 Flash Lite | Google | $0.25 | $1.50 |
| Gemma 4 31B | Google | $0.12 | $0.36 |
| GPT-4o-mini | OpenAI | $0.15 | $0.60 |
| GPT-5.4 mini | OpenAI | $0.75 | $4.50 |
| GPT-5.4 nano | OpenAI | $0.20 | $1.25 |
| Grok 4.3 | xAI | $1.25 | $2.50 |&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The Dataset 📋&lt;/h2&gt;
&lt;p&gt;8 real news articles (4 English from BBC and the Guardian, 4 Turkish from TRT and DW), hand-labeled over title + first 1,400 characters of body.&lt;/p&gt;
&lt;p&gt;That gave me &lt;strong&gt;96 gold entities&lt;/strong&gt;: 20 persons, 36 organizations, 40 locations.&lt;/p&gt;
&lt;p&gt;Labeling rules were strict and given to every model in the prompt:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Strip Turkish case suffixes.&lt;/strong&gt; &lt;em&gt;Bozbey&apos;in&lt;/em&gt; → &lt;em&gt;Bozbey&lt;/em&gt;. Period.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Demonyms map to the place.&lt;/strong&gt; &quot;Indonesian island&quot; → &lt;em&gt;Indonesia&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;No embedded entities.&lt;/strong&gt; &quot;Bursa Cumhuriyet Başsavcılığı&quot; does not also yield a separate &lt;em&gt;Bursa&lt;/em&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Acronym pairs are one entity.&lt;/strong&gt; HSK and &quot;Hakimler ve Savcılar Kurulu&quot; are the same thing; either surface form is a match.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The dataset is small — this is a first pass. Boundary judgment calls affect every model equally, so the comparison stays fair even if the absolute numbers have residual label noise.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;What I Measured 📐&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;One request per article per model, identical prompt (extract entities, return &lt;code&gt;{&quot;entities&quot;: [{&quot;name&quot;, &quot;type&quot;}]}&lt;/code&gt;, canonical form, no duplicates, temperature 0)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Micro precision / recall / F1&lt;/strong&gt; over all 96 gold entities&lt;/li&gt;
&lt;li&gt;Same metrics split by language (tr / en separately)&lt;/li&gt;
&lt;li&gt;Per-article cost at listed OpenRouter prices&lt;/li&gt;
&lt;li&gt;Mean latency per request (wall-clock)&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Throughput&lt;/strong&gt;: completion tokens per second&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;&lt;strong&gt;Decision rule, written before running:&lt;/strong&gt; highest overall micro-F1 wins. Ties within 0.03 F1 are broken by Turkish F1, then by cost, then by throughput. A model that can&apos;t return parseable JSON on any article scores 0 recall for that article — JSON reliability is part of the result.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The Results 📊&lt;/h2&gt;
&lt;p&gt;| Model | P | R | F1 | tr F1 | en F1 | Cost/8 | ms/req | tok/s |
|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 Flash 🏆 | 0.93 | 0.98 | &lt;strong&gt;0.95&lt;/strong&gt; | &lt;strong&gt;0.92&lt;/strong&gt; | 0.97 | $0.0077 | 2,434 | 135 |
| Gemini 3.5 Flash | 0.90 | 0.99 | 0.95 | 0.90 | 0.97 | $0.1830 | 12,456 | 198 |
| DeepSeek V4 Flash | 0.95 | 0.94 | 0.94 | 0.91 | 0.96 | $0.0030 | 21,081 | 77 |
| Grok 4.3 | 0.94 | 0.94 | 0.94 | 0.88 | 0.98 | $0.0235 | 5,041 | 175 |
| MiniMax M3 | 0.91 | 0.96 | 0.93 | 0.86 | 0.98 | $0.0171 | 30,125 | 53 |
| DeepSeek V4 Pro | 0.93 | 0.94 | 0.93 | 0.91 | 0.95 | $0.0201 | 41,021 | 63 |
| Gemini 3.1 Flash Lite | 0.89 | 0.96 | 0.92 | 0.85 | 0.97 | $0.0032 | 2,610 | 72 |
| Gemma 4 31B | 0.87 | 0.95 | 0.91 | 0.83 | 0.96 | $0.0015 | 14,313 | 24 |
| MiMo-V2.5-Pro | 0.88 | 0.93 | 0.90 | 0.86 | 0.93 | $0.0240 | 76,354 | 40 |
| MiMo-V2.5 | 0.87 | 0.88 | 0.87 | 0.76 | 0.95 | $0.0073 | 34,091 | 85 |
| GPT-4o-mini | 0.88 | 0.73 | 0.80 | 0.83 | 0.77 | $0.0012 | 2,538 | 55 |
| GPT-5.4 nano | 0.73 | 0.86 | 0.79 | 0.76 | 0.82 | $0.0029 | 2,651 | 79 |
| GPT-5.4 mini | 0.70 | 0.82 | 0.76 | 0.68 | 0.82 | $0.0089 | 2,237 | 75 |
| Tencent Hy3 Preview | 0.95 | 0.43 | 0.59 | 0.64 | 0.55 | $0.0094 | 100,047 | 55 |&lt;/p&gt;
&lt;p&gt;One model didn&apos;t make it across the finish line: Tencent Hy3 Preview hit the 180-second timeout on 4 of its 8 requests. When it did respond, precision was a sharp 0.95 — but a model that times out on 50% of articles is not a model you can ship.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;What Stood Out 👀&lt;/h2&gt;
&lt;h3&gt;Turkish still separates the field&lt;/h3&gt;
&lt;p&gt;English F1 runs 0.93–0.98 for most competitive models — it&apos;s mostly noise at this point. Turkish F1 spans 0.68–0.91. That gap is the whole experiment.&lt;/p&gt;
&lt;p&gt;The hardest article was an Ankara judiciary piece with 12 gold entities. No model cleanly recovered all of &quot;Ankara Cumhuriyet Başsavcılığı&quot;, &quot;Ankara Adliyesi&quot; (as an organization, not a location), and &quot;WhatsApp&quot; together. This is the canonical hard case: long institutional names with embedded place names, and a consumer brand thrown in.&lt;/p&gt;
&lt;h3&gt;The GPT-5.4 generation regressed badly 📉&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;gpt-5.4-mini&lt;/code&gt; landed second-to-last at F1 0.76 with precision collapsing to 0.70. It answers in ~2.2 seconds — apparently without any visible reasoning — and systematically over-extracts. It treats &quot;Adalet Bakanı Akın Gürlek&quot; as a single person entity (title glued to name). It extracts &quot;Bursa&quot; separately from &quot;Bursa Cumhuriyet Başsavcılığı&quot; against the explicit rules. It tags the Turkish month name &quot;Haziran&quot; as a location. The nano variant actually &lt;em&gt;beats&lt;/em&gt; the mini.&lt;/p&gt;
&lt;h3&gt;The older model beat its own successors 👻&lt;/h3&gt;
&lt;p&gt;&lt;code&gt;google/gemini-2.5-flash&lt;/code&gt; led the field at F1 0.9543 and Turkish F1 0.9250 — the top on both axes. Gemini 3.5 Flash and 3.1 Flash Lite, its direct successors, trail it on Turkish at 0.9048 and 0.85 respectively.&lt;/p&gt;
&lt;p&gt;Same day. Same prompt. Same dataset. Google&apos;s own next generation didn&apos;t beat it. &lt;strong&gt;The AI industry&apos;s newest is not always its best on specific extraction tasks.&lt;/strong&gt;&lt;/p&gt;
&lt;h3&gt;Throughput vs latency tell different stories ⚡&lt;/h3&gt;
&lt;p&gt;Grok 4.3 and Gemini 3.5 Flash stream at 175–198 tok/s and answer in under 13 seconds. DeepSeek V4 Flash manages only 77 tok/s and 21 seconds per request — reasoning overhead. Gemini 2.5 Flash sits comfortably in the middle at 135 tok/s and 2.4 seconds. For latency-sensitive applications this gap matters a lot; for batch workloads it&apos;s mostly irrelevant.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;The Verdict ✅&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;Winner: &lt;code&gt;google/gemini-2.5-flash&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;Here&apos;s how the decision played out across all three dimensions:&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Quality 🎯&lt;/strong&gt; — Gemini 2.5 Flash has the highest overall F1 (0.9543). Six candidates land within 0.03 — Gemini 3.5 Flash, DeepSeek V4 Flash, Grok 4.3, MiniMax M3, DeepSeek V4 Pro, and Gemini 3.1 Flash Lite. Turkish F1 breaks the tie: Gemini 2.5 Flash leads at 0.9250, ahead of the rest. No further tiebreaker needed.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Cost 💰&lt;/strong&gt; — $0.0077 per 8 articles ($0.00096/article). Not the cheapest in the field — DeepSeek V4 Flash is 2.6× cheaper at $0.0030 — but the winner earns its premium: it&apos;s the best F1 and Turkish F1 in the entire field, and still a fraction of the cost of premium options like Gemini 3.5 Flash ($0.1830).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Throughput ⚡&lt;/strong&gt; — 135 tok/s, 2.4 seconds per request. Among the fastest in the competitive tier. Grok 4.3 beats it on tok/s (175) but trails it on quality. Gemini 2.5 Flash is the best overall balance of speed, quality, and cost in this field.&lt;/p&gt;
&lt;p&gt;The one risk: it&apos;s a previous-generation model and the most likely to be deprecated. If that happens, &lt;strong&gt;DeepSeek V4 Flash&lt;/strong&gt; is the natural fallback — different provider, F1 0.9424, tr F1 0.9091, at $0.0030 per 8 articles.&lt;/p&gt;
&lt;hr&gt;
&lt;h2&gt;Per-Language Routing 🗺️&lt;/h2&gt;
&lt;p&gt;If you&apos;re routing NER by detected language, the data points to different winners per language:&lt;/p&gt;
&lt;p&gt;| Language | Main | Fallback 1 | Fallback 2 |
|---|---|---|---|
| 🇹🇷 Turkish | DeepSeek V4 Flash | Gemini 2.5 Flash | Grok 4.3 |
| 🇬🇧 English | Gemma 4 31B | DeepSeek V4 Flash | MiniMax M3 |&lt;/p&gt;
&lt;p&gt;Why different from the overall winner?&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Turkish main is DeepSeek V4 Flash&lt;/strong&gt;, not Gemini 2.5 Flash, because applying the same cost tiebreaker within the Turkish F1 tie window makes DeepSeek (at $0.0030) the cheapest valid option — it&apos;s 2.6× cheaper with only 0.016 less Turkish F1.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;English main is Gemma 4 31B&lt;/strong&gt; — it sits inside the English F1 tie window at 0.958, costs only $0.0015/8 articles (the cheapest on the board), and is open-weight with multi-provider availability, making it the most deprecation-resistant pick. MiniMax M3 has the top raw English F1 (0.983) but Gemma beats it on cost.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;DeepSeek V4 Flash appears in both fallback chains&lt;/strong&gt; — it performs well across both languages, which means a single model can cover both in a pinch.&lt;/li&gt;
&lt;/ul&gt;
&lt;hr&gt;
&lt;h2&gt;What I&apos;d Do Differently 🔧&lt;/h2&gt;
&lt;p&gt;The 8-article dataset is small by any serious NLP benchmark standard. It was sufficient to separate the field here — Turkish performance has enough variance that signal isn&apos;t buried in noise — but the absolute F1 numbers carry residual label noise from boundary judgment calls.&lt;/p&gt;
&lt;p&gt;A re-run is easy if the field changes: add new candidates, run the same script, compare.&lt;/p&gt;</content:encoded><h:img src="/assets/ner-llm-benchmark-gorseli.CJFly6_N.png"/><enclosure url="/assets/ner-llm-benchmark-gorseli.CJFly6_N.png"/></item><item><title>I Tested 5 AI Image Models with the Same 5 Prompts 🧪</title><link>https://osmandagdeviren.com.tr/en/blog/ai-image-models-benchmark</link><guid isPermaLink="true">https://osmandagdeviren.com.tr/en/blog/ai-image-models-benchmark</guid><description>A practical AI image model benchmark across 5 distinct capabilities — with real images, honest grades, and clear takeaways.</description><pubDate>Tue, 09 Jun 2026 12:00:00 GMT</pubDate><content:encoded>&lt;p&gt;import { benchmarkImages } from &apos;@/assets/images/ai-image-models-benchmark&apos;
import ZoomImage from &apos;@/components/custom/ZoomImage.astro&apos;&lt;/p&gt;
&lt;p&gt;I had a simple question: which image generation model actually follows instructions? 🤔&lt;/p&gt;
&lt;p&gt;Not which one produces the prettiest output. Not which one has the most impressive demo reel. Which one does what you ask, consistently, across different types of challenges.&lt;/p&gt;
&lt;p&gt;So I designed a five-prompt benchmark and ran it across five models: &lt;strong&gt;black-forest-labs/flux.2-klein-4b&lt;/strong&gt;, &lt;strong&gt;google/gemini-2.5-flash-image&lt;/strong&gt; (nano banana), &lt;strong&gt;bytedance-seed/seedream-4.5&lt;/strong&gt;, &lt;strong&gt;black-forest-labs/flux.2-pro&lt;/strong&gt;, and &lt;strong&gt;x-ai/grok-imagine-image-quality&lt;/strong&gt;. Same prompts, same grading criteria, no rerolls. 🧪&lt;/p&gt;
&lt;p&gt;This is what I found. 🔍&lt;/p&gt;
&lt;h2&gt;🧭 Why I Built This Benchmark&lt;/h2&gt;
&lt;p&gt;Most image model comparisons test the wrong thing.&lt;/p&gt;
&lt;p&gt;They ask models to generate a beautiful portrait, a fantasy landscape, or a product shot — and then declare a winner based on aesthetics. But aesthetics are not the bottleneck when you are actually trying to use these tools for real work.&lt;/p&gt;
&lt;p&gt;The real bottleneck is instruction compliance. Can the model put the right texture on the right object? Can it understand that a pencil should be &lt;em&gt;inside&lt;/em&gt; a mug, not next to it? Can it spell a word correctly on a sign? Can it override its training distribution to draw a bicycle with square wheels because that is what the prompt says?&lt;/p&gt;
&lt;p&gt;These are the things that break workflows. These are the things worth testing.&lt;/p&gt;
&lt;h2&gt;🧪 The Five Tests&lt;/h2&gt;
&lt;p&gt;I designed each prompt to stress-test a specific capability.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Test 1 — 🪵 Attribute Bleed.&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;Three objects on a white background: a textured wooden cube, a translucent blue glass sphere, a metallic gold pyramid.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The test is whether textures and materials stay bound to their correct objects, and whether the background stays clean.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Test 2 — 📐 Spatial Reasoning.&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;A desk scene: red mug on the left corner, a single yellow pencil inside the mug, an open book to the right, a potted plant on the floor under the desk.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The test is whether the model understands containment and positional relationships, not just proximity.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Test 3 — 🔤 Typography &amp;#x26; Layout.&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;A storefront with &apos;STABILITY&apos; above the door, &apos;OPEN&apos; as a neon sign in the left window, and a coffee cup drawing (no text) in the right window.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The test is spelling accuracy, correct placement, and resistance to hallucinating text where there should be none.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Test 4 — ⚙️ Structural Logic.&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;A technical blueprint of a bicycle with perfectly square wheels, showing a drive chain connecting the rear wheel hub to the foot pedals.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The test is whether the model can generate something it has almost certainly never seen in training data.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Test 5 — 🎮 Style Lockdown.&lt;/strong&gt;&lt;/p&gt;
&lt;pre&gt;&lt;code class=&quot;language-text&quot;&gt;A city intersection at rush hour, rendered as strict 8-bit pixel art with a 16-color palette, visible square pixels, no gradients, no realistic lighting.
&lt;/code&gt;&lt;/pre&gt;
&lt;p&gt;The test is whether the model maintains style discipline across a complex, detailed scene.&lt;/p&gt;
&lt;p&gt;Each test was graded as Pass (1.0), Partial Pass (0.5), or Fail (0.0).&lt;/p&gt;
&lt;h2&gt;📊 The Results&lt;/h2&gt;
&lt;p&gt;Let me go test by test.&lt;/p&gt;
&lt;h3&gt;🪵 Attribute Bleed&lt;/h3&gt;
&lt;p&gt;Four out of five models passed this cleanly. Texture separation is a solved problem for modern models—wood stayed on the cube, glass stayed on the sphere, gold stayed on the pyramid.&lt;/p&gt;
&lt;p&gt;The only deduction went to &lt;strong&gt;flux.2-pro&lt;/strong&gt;, which cast soft drop shadows under all three objects. The prompt asked for a pure white background. Shadows are not a pure white background. A minor thing, but constraint compliance is constraint compliance.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; produced the most convincing wood texture of the five—visible cracks, natural knots, the actual grain of lumber rather than a wallpaper pattern.&lt;/p&gt;
&lt;h3&gt;📐 Spatial Reasoning&lt;/h3&gt;
&lt;p&gt;This was the hardest test. Potted plant under the desk? Everyone got that. Broad-scale positional reasoning seems generally reliable.&lt;/p&gt;
&lt;p&gt;Pencil &lt;em&gt;inside&lt;/em&gt; the mug? This detail proved quite challenging for the models.&lt;/p&gt;
&lt;p&gt;Three out of five models (&lt;strong&gt;flux.2-klein-4b&lt;/strong&gt;, &lt;strong&gt;gemini-2.5-flash-image&lt;/strong&gt;, and &lt;strong&gt;seedream-4.5&lt;/strong&gt;) put coffee inside the mug even though it wasn&apos;t mentioned in the prompt.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-pro&lt;/strong&gt; was the only successful model here; it managed to place the yellow pencil inside the mug without adding coffee, earning the only full pass of the round.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; had an interesting failure mode: while it successfully placed the pencil inside the mug, it generated a dark blue circular shadow around the base of the mug on the desk. Not strictly a spatial failure, but exactly the kind of structural flaw that shows up in these relation-heavy renders.&lt;/p&gt;
&lt;p&gt;Practical takeaway: if your use case requires precise containment relationships, verify manually. Even though &lt;strong&gt;flux.2-pro&lt;/strong&gt; showed unexpectedly good performance here, models generally are not completely reliable on this yet.&lt;/p&gt;
&lt;h3&gt;🔤 Typography &amp;#x26; Layout&lt;/h3&gt;
&lt;p&gt;Four out of five models spelled &quot;STABILITY&quot; correctly. All five models placed &quot;OPEN&quot; in the left window, and all five left the right window text-free.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; was the only one to fail the main constraint—spelling it &quot;STABLITY&quot; on the main sign. One missing letter. A clean, undeniable typographical error on the most prominent element in the scene.&lt;/p&gt;
&lt;p&gt;The good news is that the negative constraint worked across the board. No model hallucinated text in the right window where the prompt explicitly said there should be none. That kind of suppression seems reliable.&lt;/p&gt;
&lt;p&gt;Typography has clearly improved in recent model generations. It is no longer the catastrophic failure point it used to be. But &quot;mostly correct&quot; is not the same as &quot;always correct,&quot; and &lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; is a reminder of that.&lt;/p&gt;
&lt;h3&gt;⚙️ Structural Logic&lt;/h3&gt;
&lt;p&gt;Two out of five models drew circular wheels.&lt;/p&gt;
&lt;p&gt;Not roughly circular. Not nearly circular. Perfectly circular, with spokes and all—exactly like a bicycle wheel looks in every training image these models have ever seen.&lt;/p&gt;
&lt;p&gt;The prompt said square wheels. Explicitly. With the word &quot;perfectly&quot; in front of it.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; and &lt;strong&gt;seedream-4.5&lt;/strong&gt; entirely defaulted to the prior. &lt;strong&gt;gemini-2.5-flash-image&lt;/strong&gt; took a different path—while it did manage to draw something resembling squares with rounded corners for the wheels, it also hallucinated a completely meaningless mechanism and nonsensical letters underneath the bicycle. The model lost its structural integrity entirely while trying to break away from its training distribution.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-pro&lt;/strong&gt; and &lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; successfully captured the square wheel form. Their wheels have sharp corners, flat faces, and diagonal structural lines indicating internal geometry.&lt;/p&gt;
&lt;p&gt;However, &lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt;&apos;s output has a significant flaw. While it looks like an impressive engineering document at first glance, a closer inspection reveals that the vast majority of the text and dimension callouts consist of illegible, nonsensical symbols. The model succumbed to typographic hallucinations while trying to create a complex blueprint aesthetic. Therefore, it only earns a partial pass.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-pro&lt;/strong&gt; shines here. Despite having no text on it, it delivers a clean, structurally accurate drawing, making it the only truly usable option among the candidates for a real-world project.&lt;/p&gt;
&lt;p&gt;What this test shows is real. When a prompt asks for something that conflicts with the visual training distribution — something the model has never seen rendered normally — most models default to what they know. The minority that pass are demonstrating something meaningfully different: the ability to construct novel structures from first principles rather than recombining familiar patterns.&lt;/p&gt;
&lt;h3&gt;🎮 Style Lockdown&lt;/h3&gt;
&lt;p&gt;Two models passed, three partially failed.&lt;/p&gt;
&lt;p&gt;The divide is stark. &lt;strong&gt;gemini-2.5-flash-image&lt;/strong&gt; and &lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; maintained hard-edged pixel discipline across the entire canvas—cars, buildings, pedestrians, signs, everything. No anti-aliasing. No gradients. Visible square pixel blocks. A limited flat color palette.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; and &lt;strong&gt;seedream-4.5&lt;/strong&gt; produced something that looks &lt;em&gt;like&lt;/em&gt; pixel art from a distance but falls apart under close inspection. Smooth car edges. Soft shadows. Road gradients. A color palette that clearly never met a 16-color limit. Furthermore, &lt;strong&gt;seedream-4.5&lt;/strong&gt;&apos;s output contains logical errors; if you look closely, there is a person walking on the roof of a building.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-pro&lt;/strong&gt;, while generally maintaining the pixel style very well, suffered from bizarre logical hallucinations. There is both a walking person and a car on the roof of a building, plus intertwined cars and people in the traffic below. Even though the visual style is solid, the scene logic collapses, preventing it from earning a full pass.&lt;/p&gt;
&lt;p&gt;What is interesting about &lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; is that it leaned entirely into the diegetic reality of the scene. The signs in the image read &quot;8-BIT BURGERS&quot;, &quot;PIXEL MART&quot;, &quot;MAIN ST / 5TH AVE&quot;. A car has a speech bubble reading &quot;HONK!&quot;. The sky is a single flat color. Every character is the same chunky four-color sprite. The model didn&apos;t just apply a pixel filter to a realistic scene — it generated a scene that &lt;em&gt;is&lt;/em&gt; a video game, from the inside.&lt;/p&gt;
&lt;p&gt;That is a different kind of style compliance.&lt;/p&gt;
&lt;h2&gt;🏁 Final Scores&lt;/h2&gt;
&lt;p&gt;| | &lt;code&gt;flux.2-klein-4b&lt;/code&gt; | &lt;code&gt;gemini-2.5-flash-image&lt;/code&gt; | &lt;code&gt;seedream-4.5&lt;/code&gt; | &lt;code&gt;flux.2-pro&lt;/code&gt; | &lt;code&gt;grok-imagine-image-quality&lt;/code&gt; |
|---|:---:|:---:|:---:|:---:|:---:|
| 🪵 Attribute Bleed | ✅ 1.0 | ✅ 1.0 | ✅ 1.0 | ⚠️ 0.5 | ✅ 1.0 |
| 📐 Spatial Reasoning | ⚠️ 0.5 | ⚠️ 0.5 | ⚠️ 0.5 | ✅ 1.0 | ⚠️ 0.5 |
| 🔤 Typography &amp;#x26; Layout | ⚠️ 0.5 | ✅ 1.0 | ✅ 1.0 | ✅ 1.0 | ✅ 1.0 |
| ⚙️ Structural Logic | ❌ 0.0 | ❌ 0.0 | ❌ 0.0 | ✅ 1.0 | ⚠️ 0.5 |
| 🎮 Style Lockdown | ⚠️ 0.5 | ✅ 1.0 | ⚠️ 0.5 | ⚠️ 0.5 | ✅ 1.0 |
| &lt;strong&gt;Total&lt;/strong&gt; | &lt;strong&gt;2.5 / 5&lt;/strong&gt; | &lt;strong&gt;3.5 / 5&lt;/strong&gt; | &lt;strong&gt;3.0 / 5&lt;/strong&gt; | &lt;strong&gt;4.0 / 5&lt;/strong&gt; | &lt;strong&gt;4.0 / 5&lt;/strong&gt; |&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-pro&lt;/strong&gt; shares the top spot with a 4.0/5. It lost a half-point on the attribute bleed test due to a minor shadow constraint violation, and another half-point in the style lockdown test due to logical scene hallucinations like a car and people on a roof.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt; also shares the top spot with a 4.0/5. Alongside a minor flaw in spatial reasoning, it lost points in the structural logic test by generating nonsensical text on its blueprint.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;gemini-2.5-flash-image&lt;/strong&gt; came in third with a 3.5/5—struggling in spatial reasoning and failing completely in the structural logic test by hallucinating a meaningless structure.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;seedream-4.5&lt;/strong&gt; scored a 3.0/5. It produced some of the most visually beautiful images in the test—the wireframe bicycle is visually stunning—but visual quality and instruction compliance are independent dimensions, and this test measures the latter.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; finished last with a 2.5/5, suffering a spelling error in typography and a clean failure in structural logic.&lt;/p&gt;
&lt;h2&gt;💰 Price Comparison&lt;/h2&gt;
&lt;p&gt;Performance is only half the story. The other half is cost. Here is how the models stack up on price per generated image:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;black-forest-labs/flux.2-klein-4b&lt;/code&gt;&lt;/strong&gt;: $0.014/image&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;black-forest-labs/flux.2-pro&lt;/code&gt;&lt;/strong&gt;: $0.03/image&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;google/gemini-2.5-flash-image&lt;/code&gt; (nano banana)&lt;/strong&gt;: $0.04/image&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;bytedance-seed/seedream-4.5&lt;/code&gt;&lt;/strong&gt;: $0.04/image&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;&lt;code&gt;x-ai/grok-imagine-image-quality&lt;/code&gt;&lt;/strong&gt;: $0.05/image&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The pricing landscape reveals some interesting dynamics. The &lt;strong&gt;flux.2-klein-4b&lt;/strong&gt; model is by far the cheapest, but it also placed last in the benchmark. Its big brother, &lt;strong&gt;flux.2-pro&lt;/strong&gt;, offers a very competitive price at $0.03/image while landing a solid 4.0/5 that ties for the top spot, making it a strong value proposition.&lt;/p&gt;
&lt;p&gt;Two of the higher-priced models—&lt;strong&gt;gemini-2.5-flash-image&lt;/strong&gt; and &lt;strong&gt;grok-imagine-image-quality&lt;/strong&gt;—sit at the higher end of the pricing spectrum at $0.04 and $0.05 per image, respectively. While both models show potential, their test results suggest they struggle to fully justify their premium price tags. &lt;strong&gt;seedream-4.5&lt;/strong&gt; matches the Gemini $0.04 price point, but scored lower on instruction compliance despite its high aesthetic quality.&lt;/p&gt;
&lt;h2&gt;🧠 What This Actually Means&lt;/h2&gt;
&lt;p&gt;A few things that I think matter beyond the numbers.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;🌀 The training distribution problem is real.&lt;/strong&gt; Circular wheels are so deeply embedded in every bicycle image these models have ever processed that &quot;draw a bicycle with square wheels&quot; is not a simple instruction — it is a battle against prior knowledge. Three out of five models lost that battle. This pattern will repeat for anything that looks like a known object but needs to be meaningfully different.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;🎨 Aesthetic quality is not the same as utility.&lt;/strong&gt; &lt;strong&gt;seedream-4.5&lt;/strong&gt;&apos;s wireframe bicycle is the most beautiful image in the structural logic test. It is also a complete failure of the test. If you are evaluating models for real work, run the prompts from your actual use case, not the ones that produce impressive portfolio pieces.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;📐 Spatial containment is a challenging detail.&lt;/strong&gt; Most models struggled with the &quot;pencil inside mug&quot; detail, defaulting to familiar patterns and adding coffee to the mug. This matters for anyone trying to generate accurate spatial diagrams, instructional illustrations, or scenes where object relationships are semantically meaningful.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;📏 The gap between first and last place is large.&lt;/strong&gt; 2.5 versus 4.0 out of 5 is not a marginal difference. The models at the top of this benchmark are doing something meaningfully different from the models at the bottom.&lt;/p&gt;
&lt;h2&gt;🔁 The Benchmark Is Reusable&lt;/h2&gt;
&lt;p&gt;These five prompts are designed to be reproducible. If you want to run the same tests on a different model, the prompts are exactly as described above, and the grading criteria are explicit.&lt;/p&gt;
&lt;p&gt;Run them. Compare the results. Share what you find. 🌍&lt;/p&gt;
&lt;p&gt;That is the kind of knowledge that becomes more useful when it is shared. 🤝&lt;/p&gt;</content:encoded><h:img src="/assets/yz-gorsel-modelleri-karsilastirmasi.DZCedvQv.png"/><enclosure url="/assets/yz-gorsel-modelleri-karsilastirmasi.DZCedvQv.png"/></item><item><title>Hello World</title><link>https://osmandagdeviren.com.tr/en/blog/hello-world</link><guid isPermaLink="true">https://osmandagdeviren.com.tr/en/blog/hello-world</guid><description>The first note of my personal technical archive: a small beginning for everything I build, break, learn, and document.</description><pubDate>Sat, 06 Jun 2026 06:30:00 GMT</pubDate><content:encoded>&lt;p&gt;Every project needs a first output.&lt;/p&gt;
&lt;p&gt;For some, it is a blinking LED.&lt;br&gt;
For others, it is a clean terminal prompt, a successful build, a green CI pipeline, or a small web page that finally renders without errors.&lt;/p&gt;
&lt;p&gt;For this website, it is this post.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Hello World.&lt;/strong&gt;&lt;/p&gt;
&lt;p&gt;This is the first note on osmandagdeviren.com.tr, and I want it to stay simple. No exaggerated promises, no perfectly polished introduction, no attempt to make this place look bigger than it is on day one. This website starts as a personal space: a place where I can write, document, test ideas, keep technical notes, and leave useful traces for my future self.&lt;/p&gt;
&lt;p&gt;If those traces help someone else along the way, even better.&lt;/p&gt;
&lt;h2&gt;Why this website exists&lt;/h2&gt;
&lt;p&gt;I have always liked the idea of owning a small corner of the internet.&lt;/p&gt;
&lt;p&gt;Not just a profile page on someone else&apos;s platform. Not just scattered notes across chats, repositories, bookmarks, and forgotten text files. I wanted a place that feels closer to a technical notebook: something I can shape, break, fix, redesign, and extend over time.&lt;/p&gt;
&lt;p&gt;This website is not meant to be a perfect knowledge base from the beginning. It is more like a living archive.&lt;/p&gt;
&lt;p&gt;Some posts may be structured guides.&lt;br&gt;
Some may be debugging notes.&lt;br&gt;
Some may be quick reminders.&lt;br&gt;
Some may be opinionated experiments.&lt;br&gt;
Some may simply document what worked, what failed, and what I learned while trying to make something work.&lt;/p&gt;
&lt;p&gt;That is the point.&lt;/p&gt;
&lt;p&gt;Real technical work is rarely as clean as final documentation makes it look. Systems fail. Commands behave differently than expected. Drivers break. Containers refuse to start. Models do not fit into memory. A configuration line that looks harmless can waste hours. Sometimes the final answer is elegant; sometimes it is just a tiny fix hidden inside a long trail of errors.&lt;/p&gt;
&lt;p&gt;I want this site to preserve that trail.&lt;/p&gt;
&lt;h2&gt;What I will write about&lt;/h2&gt;
&lt;p&gt;The topics here will probably move around the things I already spend time with: Linux, self-hosting, homelabs, Docker, Proxmox, automation, web development, artificial intelligence models, performance tests, hardware experiments, and all the small technical details that become important only after something breaks.&lt;/p&gt;
&lt;p&gt;There will be posts about tools I use.&lt;br&gt;
There will be posts about systems I build.&lt;br&gt;
There will be posts about mistakes I make.&lt;br&gt;
There will be posts that are mostly notes to myself.&lt;/p&gt;
&lt;p&gt;I do not want every article to sound like a tutorial written from a distance. I prefer writing closer to the real process: what I tried, what happened, what confused me, how I investigated it, and what finally solved the problem.&lt;/p&gt;
&lt;p&gt;That kind of writing may not always be the shortest path to an answer, but it is often the most useful path when you are dealing with the same problem in the real world.&lt;/p&gt;
&lt;h2&gt;A personal technical archive&lt;/h2&gt;
&lt;p&gt;One of the main reasons for starting this blog is simple: I forget things.&lt;/p&gt;
&lt;p&gt;Not the big ideas, but the exact commands. The small flags. The order of operations. The workaround that fixed a weird issue at 03:00. The reason I chose one tool instead of another. The context behind a configuration file that looked obvious when I wrote it.&lt;/p&gt;
&lt;p&gt;Writing things down turns temporary solutions into reusable knowledge.&lt;/p&gt;
&lt;p&gt;So this website is also an attempt to build my own memory layer. A searchable, public, versioned place for the things I learn while working with software, hardware, infrastructure, and automation.&lt;/p&gt;
&lt;p&gt;Maybe some posts will age badly. Maybe some solutions will become obsolete. That is fine. Technology changes, and a good archive should show that change instead of pretending everything is timeless.&lt;/p&gt;
&lt;h2&gt;Shared knowledge&lt;/h2&gt;
&lt;p&gt;I believe knowledge becomes more valuable when it is shared.&lt;/p&gt;
&lt;p&gt;The notes, guides, experiments, and explanations published on this blog are not meant to stay locked inside a private archive. They are here to be read, used, adapted, discussed, improved, and carried forward by others, as long as the original work is respected.&lt;/p&gt;
&lt;p&gt;That is why the blog posts on this website are published under the &lt;strong&gt;Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License&lt;/strong&gt; unless stated otherwise.&lt;/p&gt;
&lt;p&gt;In simple terms: you can share and adapt the content, but you should give proper credit, avoid commercial use, and keep derivative works under the same license.&lt;/p&gt;
&lt;p&gt;This feels aligned with the kind of internet I want to contribute to: an internet where useful knowledge does not disappear inside closed platforms, private notes, or forgotten chat logs, but keeps growing through people who learn from each other.&lt;/p&gt;
&lt;p&gt;If something I write here saves someone a few hours, helps them understand a problem, or gives them a starting point for their own work, then this blog has already done something useful.&lt;/p&gt;
&lt;h2&gt;The tone of this place&lt;/h2&gt;
&lt;p&gt;I do not want this blog to feel overly corporate or artificially polished.&lt;/p&gt;
&lt;p&gt;I want it to feel practical, personal, and honest.&lt;/p&gt;
&lt;p&gt;When something works, I will write what worked.&lt;br&gt;
When something does not work, I will try to explain why.&lt;br&gt;
When I am unsure, I will say that I am unsure.&lt;br&gt;
When a post is just a note, it will remain a note.&lt;/p&gt;
&lt;p&gt;The goal is not to look like I have everything figured out. The goal is to keep learning in public and to build a useful technical archive over time.&lt;/p&gt;
&lt;h2&gt;First commit&lt;/h2&gt;
&lt;p&gt;A “Hello World” is usually not impressive by itself.&lt;/p&gt;
&lt;p&gt;It does not solve a major problem. It does not prove that the architecture is good. It does not guarantee that the project will become useful. It simply proves that the first connection has been made.&lt;/p&gt;
&lt;p&gt;The system runs.&lt;br&gt;
The page exists.&lt;br&gt;
The first output is visible.&lt;/p&gt;
&lt;p&gt;That is enough for a beginning.&lt;/p&gt;
&lt;p&gt;So this post can stay here as the first commit of this website’s written archive.&lt;/p&gt;
&lt;p&gt;Hello World.&lt;/p&gt;</content:encoded><h:img src="/assets/hello-world.3FALAXBW.png"/><enclosure url="/assets/hello-world.3FALAXBW.png"/></item></channel></rss>