Path Interpretations¶

GFM-RAG exhibits the multi-hop reasoning ability powered by the multi-layer GFM. You can visualize the paths captured by the GFM-RAG to provide path interpretations for the prediction.

The paths' importance to the final prediction can be quantified by the partial derivative of the prediction score with respect to the triples at each layer (hop), defined as:

\[ s_1,s_2,\ldots,s_L=\arg\mathop{\text{top-}}k\frac{\partial p_e(q)}{\partial s_l} \]

The top-\(k\) path interpretations can be obtained by the top-\(k\) longest paths with beam search.

You can visualize the paths and their importance using the following code:

gfmrag/workflow/config/exp_visualize_path.yaml

hydra:
  run:
    dir: outputs/experiments/visualize/${dataset.data_name}/${now:%Y-%m-%d}/${now:%H-%M-%S}

defaults:
  - _self_
  - doc_ranker: idf_topk_ranker

seed: 1024

dataset:
  root: ./data
  data_name: hotpotqa_test

graph_retriever:
  model_path: rmanluo/GFM-RAG-8M

test:
  retrieval_batch_size: 8
  top_k: 5
  save_retrieval: False
  save_top_k_entity: 10
  n_threads: 5
  retrieved_result_path: null
  prediction_result_path: null
  init_entities_weight: True
  max_sample: 10

gfmrag/workflow/experiments/visualize_path.py

--8<--"gfmrag/workflow/experiments/visualize_path.py"

Bash

python -m gfmrag.workflow.experiments.visualize_path

Path Examples

Text Only

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
Question: What football club was owned by the singer of "Grow Some Funk of Your Own"?
Answer: Watford Football Club
Question Entities: ['grow some funk of your own', 'football club']
Supporting Facts: ['Grow Some Funk of Your Own', 'Elton John']
Predicted Paths:
--------------------------------------------------------
Target Entity: watford football club
1.0950: [ grow some funk of your own, is a song by, elton john ] => [ elton john, equivalent, sir elton hercules john ] => [ sir elton hercules john, inverse_named a stand after, watford football club ]
0.9154: [ grow some funk of your own, is a song by, elton john ] => [ elton john, equivalent, sir elton hercules john ] => [ sir elton hercules john, owned, watford football club ]