Hello everyone, I have some questions regarding Hybrid Tree Search and would appreciate your insights. #175

Activity

1. denis-samatov commented on Jul 9, 2026

   

   denis-samatov

   on Jul 9, 2026

   More actions

   Thanks for the question. I would treat this diagram as a hybrid extension around PageIndex rather than the default PageIndex retrieval mode.

   PageIndex itself is mainly a reasoning/tree-based retrieval system: it builds a document tree / table-of-contents-like structure and then uses reasoning over that structure to find relevant sections. If you add a vector-based node retriever in parallel, I would use it as a high-recall candidate generator, not as the final decision maker.

   1. Parallel Retrieval: how many nodes should the vector-based retriever return?

   There is no universal best fixed number. The right value depends on:

   • document length;
   • average node size;
   • query complexity;
   • token budget;
   • whether the query is single-hop or multi-hop;
   • whether you have a reranking / filtering stage after retrieval.

   A practical starting point would be:

   • simple factual query: top_k = 5–10;
   • normal document QA: top_k = 10–20;
   • long document or multi-hop query: top_k = 20–40 candidates;
   • final answer context: usually keep only 3–8 strong nodes after filtering.

   I would not send all vector-retrieved nodes directly to the final answer generator. A safer pipeline is:

   1. retrieve broad candidates with vector search;
   2. retrieve structure-aware candidates with reasoning/tree search;
   3. merge candidates;
   4. deduplicate by node_id, page range, or section title;
   5. rerank/filter them by query relevance;
   6. pass only the strongest evidence nodes to the node consumer / final answer step.

   For example:

   def choose_vector_top_k(query_type: str) -> int:
       if query_type == "narrow_fact":
           return 8
       if query_type == "multi_hop" or query_type == "broad":
           return 25
       return 15
   
   Then apply a token-budget filter before calling the LLM.
   
   The important point is that vector retrieval should optimize recall, while the reasoning/tree retriever and node consumer should optimize precision.
   
   2. Node Consumer: when to extract and when to summarize?
   
   I would separate extraction and summarization into two different operations.
   
   Use extraction when:
   the user asks for exact facts, numbers, dates, names, requirements, definitions, or citations;
   the node is small enough to fit into the context window;
   the final answer needs precise evidence;
   traceability to page/section/node is important.
   
   In this case, the node consumer should return structured evidence, for example:
   
   {
     "node_id": "...",
     "page_range": "...",
     "relevant_facts": ["..."],
     "supporting_quotes": ["..."],
     "relevance_score": 0.0,
     "is_sufficient": true
   }
   Use summarization when:
   the node is too large to pass downstream directly;
   the user asks a broad question;
   several nodes must be compressed before final reasoning;
   the node is used as intermediate context rather than final evidence.
   
   However, I would avoid generic summaries. The summary should be query-focused.
   
   For example, instead of:
   
   Summarize this node.
   
   I would use:
   
   Summarize only the information in this node that is relevant to the user query.
   Preserve numbers, entities, dates, page references, and uncertainty.
   Do not add information that is not present in the node.
   
   A robust node consumer flow could look like this:
   
   for node in candidate_nodes:
       if node.token_count <= NODE_DIRECT_READ_LIMIT:
           evidence = extract_relevant_facts(node, query)
       else:
           compressed_node = summarize_node_for_query(node, query)
           evidence = extract_relevant_facts(compressed_node, query)
   
       if evidence.is_relevant:
           evidence_pool.append(evidence)
   
   Then the final answer generator should answer only from evidence_pool, not from all raw retrieved nodes.
   
   Suggested mental model
   Vector Retriever     = broad semantic recall
   Reasoning Retriever  = structure-aware navigation
   Node Consumer        = query-focused extraction / compression
   Final Answer         = grounded synthesis from selected evidence
   
   So my recommendation is:
   
   start with top_k = 10–20 for vector retrieval;
   increase to 20–40 for long or multi-hop questions;
   rerank/filter and keep only 3–8 nodes for final synthesis;
   extract when precision is required;
   summarize only when the node is too large or the question is broad;
   prefer query-focused summaries over generic summaries.
2. ctlove0523 commented on Jul 9, 2026

   

   ctlove0523

   on Jul 9, 2026 via email

   Author

   More actions

   你好，你的邮件我已经收到，每天会在晚上查看邮件并回复。急事请打电话