These docs describe changes that aren’t released yet. For the newest release, see the released docs.
ChunkOptions
How the Markdown is cut into chunks.
Properties
Section titled “Properties”maxTokens?
Section titled “maxTokens?”
readonlyoptionalmaxTokens?:number
A chunk’s budget in approximate tokens (characters / 4); 512 by default. Blocks are never cut.
overlap?
Section titled “overlap?”
readonlyoptionaloverlap?:number
Approximate tokens of whole blocks a chunk repeats from the one before it; 0 by default.
splitOn?
Section titled “splitOn?”
readonlyoptionalsplitOn?:any
Where chunks may start.