tree_sitter_python_external_scanner_serialize writes the indent stack in a loop guarded by size < TREE_SITTER_SERIALIZATION_BUFFER_SIZE, then writes two bytes per iteration. When the header before the loop has odd length (2 + delimiter_count, so one open string delimiter), size can reach 1023. The last iteration then writes one byte past the 1024-byte buffer and returns 1025, and the runtime aborts:
Assertion failed: (length <= 1024), function ts_parser__external_scanner_serialize, file parser.c, line 409.
|
uint32_t iter = 1; |
|
for (; iter < scanner->indents.size && size < TREE_SITTER_SERIALIZATION_BUFFER_SIZE; ++iter) { |
|
uint16_t indent_value = *array_get(&scanner->indents, iter); |
|
buffer[size++] = (char)(indent_value & 0xFF); |
|
buffer[size++] = (char)((indent_value >> 8) & 0xFF); |
|
} |
Repro (tree-sitter-python 0.25.0, tree-sitter 0.27.0 runtime, macOS arm64; src/scanner.c is byte-identical on master at 26855ea): 600 lines of if 1:, each indented one space more than the last, then 'a' as the body of the innermost one.
python3 -c "print(''.join(' ' * i + 'if 1:\n' for i in range(600)) + ' ' * 600 + \"'a'\")" > deep.py
The same shape at 400 levels parses.
Suggested fix: test for the bytes about to be written:
for (; iter < scanner->indents.size && size + 2 <= TREE_SITTER_SERIALIZATION_BUFFER_SIZE; ++iter) {
tree_sitter_python_external_scanner_serializewrites the indent stack in a loop guarded bysize < TREE_SITTER_SERIALIZATION_BUFFER_SIZE, then writes two bytes per iteration. When the header before the loop has odd length (2 + delimiter_count, so one open string delimiter),sizecan reach 1023. The last iteration then writes one byte past the 1024-byte buffer and returns 1025, and the runtime aborts:tree-sitter-python/src/scanner.c
Lines 382 to 387 in 26855ea
Repro (tree-sitter-python 0.25.0, tree-sitter 0.27.0 runtime, macOS arm64;
src/scanner.cis byte-identical on master at 26855ea): 600 lines ofif 1:, each indented one space more than the last, then'a'as the body of the innermost one.The same shape at 400 levels parses.
Suggested fix: test for the bytes about to be written: