Skip to content

[Python] Remove eval() from parse_chunk_strings and move it to RAG test_utils - #41403

Draft
tvalentyn wants to merge 1 commit into
apache:masterfrom
tvalentyn:remove-eval-parse-chunk-strings
Draft

tvalentyn wants to merge 1 commit into
apache:masterfrom
tvalentyn:remove-eval-parse-chunk-strings

Conversation

@tvalentyn

Copy link
Copy Markdown
Contributor

No description provided.

…st utils

apache_beam.ml.rag.utils.parse_chunk_strings turned printed Chunks back
into objects with eval() and an empty __builtins__. That restriction does
not stop attribute access such as ().__class__.__base__, so a caller that
passed it untrusted strings could run arbitrary code. Its only caller is
the Milvus enrichment example test.

Remove the function from utils.py and add a replacement to
apache_beam/ml/rag/test_utils.py that parses the string with ast and
builds the value from the syntax tree. Only literals, lists, tuples, dicts
and the Chunk, EmbeddableItem, Content, Embedding and defaultdict
constructors are accepted; nothing is evaluated.

The replacement also accepts EmbeddableItem, which is what repr() of a
Chunk prints since Chunk became an alias. The old helper could not parse
that output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant