bangla_pdf 1.3.0

Correct Bangla (Bengali) text in PDFs: real OpenType shaping for conjuncts, reph and vowel signs, with text that copies back out as clean Unicode.

bangla_pdf

Correct Bangla in PDFs.
Real OpenType shaping for conjuncts, reph and vowel signs — and text that copies back out as clean Unicode.

---

package:pdf has no complex-script shaping, so Bangla comes out broken: conjuncts fall apart, ি ে ৈ land on the wrong side of their consonant, and reph (র্) goes anywhere but where it belongs.

bangla_pdf shapes Bangla with the font's own OpenType GSUB/GPOS tables, in pure Dart — no FFI, no native toolchain, works on web. Drop in Text() and it just works.

🧬 Real shaping	The font's own GSUB/GPOS, not a lookup table. Conjuncts, reph, phalas and pre-base matras all correct.
📋 Copy-pasteable	The PDF holds real Unicode. pdftotext, search and screen readers get your original string back.
🎯 Zero config	No initialisation, no asset bundling. A Bangla font ships with the package.
🪶 Pure Dart	No FFI, no C toolchain, no native build step — so it works on web too.
🔒 Drop-in	Same parameters as the package:pdf widgets — swap pw.Text for Text and nothing else changes.
📥 Reads PDFs too	extract.dart pulls Bangla back out — including Bijoy documents, converted to Unicode.
🧪 Verified	Diffed glyph-by-glyph against HarfBuzz: 234/234 on five fonts.

---

🚀 Quick start

dependencies:
  bangla_pdf: ^1.3.0

import 'package:bangla_pdf/bangla_pdf.dart';
import 'package:pdf/widgets.dart' as pw;

final pdf = pw.Document();

pdf.addPage(
  pw.Page(
    build: (context) => Text('আমার সোনার বাংলা, আমি তোমায় ভালোবাসি।'),
  ),
);

final bytes = await pdf.save();

No initialisation, no font registration, no asset bundling. A Bangla font ships with the package and is used automatically.

---

✨ What it fixes

	before (1.0.6)	now
কর্ম ধর্ম বর্ষ পূর্ব শর্ত	threw RangeError, killed pdf.save()	render correctly
ক্ষ্ম ঙ্ক্ষ ত্ত্ব চ্ছ্ব ম্ভ্র স্ত্র্য	base + a stray hasanta ্	ligated
copy/paste, search, pdftotext	Avgvi ‡mvbvi evsjv	the original Unicode
Bengali digits ০–৯	silently became 0–9	preserved
৳	silently became $	preserved
ৰ ৱ ॥	missing glyph	rendered
র‍্য vs র্য (ZWJ)	ZWJ discarded, both identical	distinguished
NFC vs NFD input	mostly equivalent	identical by construction

Measured, not asserted — every row is checked by the test suite.

📋 The text in the PDF is real Bangla

Run pdftotext over a document generated by each version. 1.0.6:

$ pdftotext invoice.pdf -
Avgvi ‡mvbvi evsjv , Avwg ‡Zvgvq fv‡jvevwm| ¶&g O&¶ Ë&e ”Q&e g&å

Not one Bengali codepoint — the PDF stored Bijoy ANSI, so copy, search, grep and screen readers all got mojibake. 1.1.0:

$ pdftotext invoice.pdf -
আমার সোনার বাংলা, আমি তোমায় ভালোবাসি।
ক্ষ্ম ঙ্ক্ষ ত্ত্ব চ্ছ্ব ম্ভ্র স্ত্র্য

Byte-for-byte the string you passed to Text().

---

🧩 Widgets

Every widget takes a plain Dart String in logical order and handles the rest.

Text('বাংলা টেক্সট')
Header('বাংলা শিরোনাম')
Paragraph('একটি অনুচ্ছেদ।')

BulletList(items: ['প্রথম আইটেম', 'Second item', 'তৃতীয় আইটেম'])

Table(data: [
  ['পণ্য',   'পরিমাণ', 'মূল্য'],
  ['কফি',    '২',      '৳২০'],
  ['বিস্কুট', '৩',      '৳১৫'],
])

RichText(spans: [
  TextSpan('বাংলা বোল্ড ', fontWeight: pw.FontWeight.bold),
  TextSpan('এবং সাধারণ'),
])

_{Every widget on one page — all shaped, all selectable.}

Mixed scripts need no special handling — Latin, Bengali digits and currency all render in one pass:

Text('Invoice #1042 - মোট ৳১২,৫০০.০০ - তারিখ ০১/০৯/২০২৬')

Drop-in for package:pdf

Each widget takes the same parameters as its pw counterpart, so switching is a one-word change:

- pw.Text('বাংলা', style: pw.TextStyle(fontSize: 18), maxLines: 2)
+    Text('বাংলা', style: pw.TextStyle(fontSize: 18), maxLines: 2)

Header, Paragraph, RichText, TextSpan and BulletList accept their pw equivalents' parameters, and Table mirrors pw.TableHelper.fromTextArray — headerCount, headerDecoration, oddRowDecoration, cellAlignments, columnWidths and the rest.

Three differences worth knowing:

• Header and Paragraph take their text positionally (Header('শিরোনাম')), where pw takes it as text:. That is how this package has worked since 1.0 and changing it would break every existing call.
• Paragraph defaults to TextAlign.start rather than pw's justify.
• One font draws the whole string, so style and banglaStyle no longer give Bangla and Latin different looks inside one widget — banglaStyle wins when both are set. Use two widgets if you want two looks.

AutoText and RichTextItem are deprecated. Text is identical to AutoText — splitting a string by script stopped being necessary once a Unicode Bangla font covered Latin and digits too — and RichTextItem was never used by anything. Both still work and are removed in 2.0.0.

Full Text parameters

Text(
  'বাংলা',
  fontSize: 16,
  fontWeight: pw.FontWeight.normal,
  color: PdfColors.black,
  textAlign: pw.TextAlign.start,
  maxLines: 3,
  banglaFont: myFont,               // font for Bangla runs
  style: pw.TextStyle(...),         // non-Bangla runs
  banglaStyle: pw.TextStyle(...),   // Bangla runs
)

---

🔤 Using your own font

final font = BanglaPdf.loadFont(
  await rootBundle.load('assets/fonts/SolaimanLipi.ttf'),
);

Text('বাংলা', banglaFont: font);          // one widget
BanglaPdf.configure(defaultFont: font);   // everywhere

BanglaPdf.loadFont returns a font that shapes with its own OpenType tables and can draw conjunct glyphs no codepoint maps to. A plain pw.Font.ttf still works but is treated as a legacy 8-bit font.

---

⚙️ Shaping modes

BanglaPdf.configure(shapingMode: BanglaShapingMode.auto);

mode	behaviour
auto (default)	shape with the font's OpenType tables; fall back to legacy for a font with no Bengali coverage
unicode	always shape, even if the font looks unsuitable
legacy	the 1.0.x Bijoy ANSI pipeline, for byte-identical old output

---

📥 Reading Bangla back out of a PDF

package:bangla_pdf/extract.dart is a separate library, so generating PDFs costs nothing if you never import it.

import 'package:bangla_pdf/extract.dart';

final result = BanglaPdfExtractor.extract(bytes);

print(result.encodingDetected);  // unicode | bijoy | mixed | none
print(result.confidence);        // 0..1
print(result.text);

It handles the three kinds of document you actually meet:

document	what happens
Unicode — from this package or any modern producer	read directly from /ToUnicode and /ActualText
Bijoy / ANSI — most Bangladeshi government and newspaper PDFs	the text layer is Latin-1 mojibake (Avgvi ‡mvbvi evsjv); converted back to Unicode per run, because only a run's font can tell Bijoy bytes from real English
Scanned	reported as BanglaTextEncoding.none rather than guessed at

Scanned pages

No OCR engine is bundled. Wire in whichever you already use:

final result = BanglaPdfExtractor.extract(
  bytes,
  ocrHook: (page) => runTesseract(page.number, language: 'ben'),
);

Converting Bijoy text on its own

bijoyToUnicode('Avgvi ‡mvbvi evsjv');   // আমার সোনার বাংলা

---

📚 API

symbol	purpose
Text Header Paragraph RichText TextSpan BulletList Table	widgets
BanglaPdf.configure({shapingMode, defaultFont})	package-wide settings
BanglaPdf.loadFont(ByteData)	load a Unicode Bangla font
BanglaPdf.covers(font, text)	can this font draw this string?
BanglaPdf.defaultFont · .shapingMode · .reset()	current configuration
BanglaShapingMode	auto · unicode · legacy
String.shapeForPdf(font) → ShapedRun	glyphs, advances, offsets and cluster text, for callers drawing their own content
BanglaFontManager().defaultFont · .legacyFont	the bundled fonts
String.fix · String.isBanglaText	1.0.x helpers, unchanged
BanglaPdfExtractor.extract(bytes, {ocrHook})	read Bangla out of a PDF (extract.dart)
ExtractionResult · ExtractedPage · BanglaTextEncoding	what extraction returns
bijoyToUnicode(String)	convert a Bijoy/ANSI string on its own

---

🔬 How it works

1. Normalise — NFC and NFD are reconciled, two-part vowels (ো ৌ) are split the way font rules expect, and nukta pairs are composed.
2. Segment the text into Indic syllables.
3. Find the base consonant by asking the font which consonants it gives below-base or post-base forms to.
4. Reorder — pre-base matras move in front of the cluster; reph moves after the base and any below-base form.
5. Apply GSUB in the OpenType Indic order — nukt akhn rphf blwf half pstf vatu cjct, then pres abvs blws psts haln, then calt clig rclt rlig — each under the right per-glyph mask.
6. Apply GPOS — dist abvm blwm mark mkmk kern, including mark-to-base, mark-to-ligature and mark-to-mark attachment.
7. Emit PDF text as a Type0/Identity-H CID font addressed by glyph id, with an explicit /CIDToGIDMap and a /ToUnicode CMap whose entries may span several codepoints — so a conjunct copies back as its full sequence.
8. Wrap each line in a /Span <</ActualText …>> BDC … EMC marked-content span. This is what makes copy/paste survive Bengali's glyph reordering: কি draws ি first, so no per-glyph mapping alone can express logical order.

Line breaking happens on syllable boundaries, so a line never breaks inside a conjunct or between a vowel sign and its consonant.

---

🧪 Verification

Every claim below is produced by code in this repository. The full write-ups live there rather than in the published package — see the verification report for the complete run, and the 1.0.6 teardown for how the old pipeline worked and why it broke.

Shaping is diffed glyph-by-glyph and position-by-position against HarfBuzz (hb-shape) over a 253-case corpus:

font	Indic spec	exact match
Kalpurush (bundled)	v2 bng2	234 / 234
SolaimanLipi	v1 beng	234 / 234
Siyam Rupali	v2 bng2	234 / 234
Noto Sans Bengali	v2 bng2	234 / 234
Noto Serif Bengali	v2 bng2	234 / 234

Both generations of the OpenType Indic spec are handled: v1 fonts write their half/blwf/pstf rules as consonant + virama and v2 fonts the other way round, so the shaper reorders to give each the order it expects.

Round-trip — every corpus case is rendered to a real PDF and extracted with pdftotext: 249 / 251 identical to the source. The two exceptions are one artefact: a lone ZWJ renders as nothing, so the line-by-line comparison shifts.

Structure — automated tests parse the generated PDF and assert it is a Type0 CID font, that every /ToUnicode entry is real Unicode with no U+FFFD or NUL, and that each line's /ActualText decodes back to its source string.

Extraction is measured against ten fixture documents shaped like the real thing — three Bijoy government notices, three invoices, two newspaper pages and two scans — each generated so its expected text is known exactly:

	result
fixtures recovering their ground truth	8 / 8 extractable, exactly
scans correctly reported as having no text layer	2 / 2
corpus surviving a Bijoy → Unicode round-trip	245 / 247 (99%)

flutter test    # 26 tests

---

⬆️ Upgrading from 1.0.x

Three things to know:

1. Nothing to change. Text(...), banglaStyle:, banglaFont: and the no-initialisation default all behave as before. No API was removed.
2. The typeface does not change. The bundled font is still Kalpurush, now its Unicode build instead of the 8-bit Bijoy one. Documents look the same; what changes is that the Bangla in them is shaped correctly.
3. Custom Bijoy fonts keep working. A legacy 8-bit font passed to banglaFont: is detected and routed to the old pipeline automatically. To force the old behaviour everywhere, call BanglaPdf.configure(shapingMode: BanglaShapingMode.legacy) once at startup.

---

⚠️ Known limitations

Stated plainly rather than glossed over.

• Extraction does not un-shape third-party Unicode PDFs. A document whose /ToUnicode CMap is missing or wrong, and which has no /ActualText, cannot be recovered: that needs reversing the shaping from glyph ids back to characters, which is not implemented.
• Encrypted PDFs are not decrypted. ExtractionResult.isEncrypted says so rather than returning nonsense.
• Extraction fixtures are generated, not collected. They are shaped like real government notices, invoices and newspaper pages, and the Bijoy ones use a genuine legacy font through the 1.0.x pipeline — but no PDF from an actual government website has been tested.
• Five fonts are measured, all matching HarfBuzz exactly: bundled Kalpurush, SolaimanLipi, Siyam Rupali, Noto Sans Bengali and Noto Serif Bengali. Others are untested. A font relying on GSUB lookup type 8 (reverse chaining) or GPOS type 3 (cursive attachment) would not shape — neither is implemented, because no tested Bengali font uses them.
• Rendering is not pixel-diffed. Output was rasterised and compared against HarfBuzz reference renders by eye for a sample, not automatically for all 253 cases.
• Copy/paste is verified with poppler only. Adobe Reader, macOS Preview, Chrome and Android viewers have not been tested, and /ActualText support does vary between them.
• The full font is embedded in every PDF (121 KB); there is no per-document subsetter yet.
• Emoji need a fallback font. The bundled font covers ASCII, the Bengali block and common punctuation. Emoji render as a placeholder box, and BulletList falls back from • to · because Kalpurush has no bullet glyph.

---

🤝 Contributing

Bug reports, corpus cases and pull requests are all welcome — a failing string is especially useful. Add it to test/corpus/bangla_cases.json with an id, category and notes, and the differential harness will pick it up.

flutter test                                  # unit and PDF-structure tests
dart run tool/dev/shape_dump.dart <font.ttf>  # diff against hb-shape
dart run tool/baseline/run_106_mapper.dart …  # the 1.0.6 baseline

hb-shape/hb-view (brew install harfbuzz) and pdftotext/pdftoppm (brew install poppler) are needed for the verification tooling, not for the package itself.

---

📄 License

BSD 3-Clause — see LICENSE.

The bundled Kalpurush is by Md. Tanbin Islam Siyam (Avro Font Development Project, omicronlab.com) under the SIL Open Font License 1.0; its Latin glyphs are from Gentium. Font licences are in LICENSE-FONTS.txt.

---

💛 Credits

Maintained by Zaman Sheikh · [e-mail]

The legacy Bijoy pipeline kept for backward compatibility (BanglaShapingMode.legacy) descends from the ANSI transcoding approach in AR Rahman's bangla_pdf_fixer 2.x. It is retained only so 1.0.x users can reproduce their old output; the shaping in this package does not use it.

Thanks to every Bangla font creator whose work makes readable Bangla typography possible.

⭐ If this saved you a day of debugging, star the repo.