【GPT入门】第30课 langchain的自问自搜索回答agent不同模型对比

概要

使用自问自搜自答agent，对比不同模型的执行效果。
gpt4：很牛
gpt3.5: 有的问题会失败
通义千问：执行失败
智谱：执行失败
结论：不同模型的大脑的真的不一样

1. 代码

from langchain import hub
from langchain.agents import AgentExecutor, create_self_ask_with_search_agent
from langchain_community.tools.tavily_search import TavilyAnswer
from langchain_openai import ChatOpenAI

# 将初始化工具，让它提供答案而不是文档
tools = [TavilyAnswer(max_results=1, name="Intermediate Answer", description="Answer Search")]

# 初始化大模型
# llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")
import os
from langchain_community.chat_models import ChatTongyi
# llm = ChatTongyi(
#     model="qwen-max",
#     api_key=os.getenv("DASHSCOPE_API_KEY"),
#     base_url=os.getenv("DASHSCOPE_BASE_URL")
# )

from langchain_community.chat_models import ChatZhipuAI
llm = ChatZhipuAI(
    model="glm-4-flash",
    api_key=os.getenv("ZHIPUAI_API_KEY"),
    base_url=os.getenv("ZHIPUAI_BASE_URL")
)
# 获取使用提示 可以修改此提示
prompt = hub.pull("hwchase17/self-ask-with-search")
print(prompt.template)
# 使用搜索代理构建自助询问
agent = create_self_ask_with_search_agent(llm, tools, prompt)

# 通过传入代理和工具创建代理执行程序
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=True)

# 运行代理
res = agent_executor.invoke({"input": "中国有哪些省份呢?"})
print('res:',res)

2 输出结果

用智谱和通义千问，执行都报错； 用GPT3.5 可以正常执行

Question: Who lived longer, Muhammad Ali or Alan Turing?
Are follow up questions needed here: Yes.
Follow up: How old was Muhammad Ali when he died?
Intermediate answer: Muhammad Ali was 74 years old when he died.
Follow up: How old was Alan Turing when he died?
Intermediate answer: Alan Turing was 41 years old when he died.
So the final answer is: Muhammad Ali

Question: When was the founder of craigslist born?
Are follow up questions needed here: Yes.
Follow up: Who was the founder of craigslist?
Intermediate answer: Craigslist was founded by Craig Newmark.
Follow up: When was Craig Newmark born?
Intermediate answer: Craig Newmark was born on December 6, 1952.
So the final answer is: December 6, 1952

Question: Who was the maternal grandfather of George Washington?
Are follow up questions needed here: Yes.
Follow up: Who was the mother of George Washington?
Intermediate answer: The mother of George Washington was Mary Ball Washington.
Follow up: Who was the father of Mary Ball Washington?
Intermediate answer: The father of Mary Ball Washington was Joseph Ball.
So the final answer is: Joseph Ball

Question: Are both the directors of Jaws and Casino Royale from the same country?
Are follow up questions needed here: Yes.
Follow up: Who is the director of Jaws?
Intermediate answer: The director of Jaws is Steven Spielberg.
Follow up: Where is Steven Spielberg from?
Intermediate answer: The United States.
Follow up: Who is the director of Casino Royale?
Intermediate answer: The director of Casino Royale is Martin Campbell.
Follow up: Where is Martin Campbell from?
Intermediate answer: New Zealand.
So the final answer is: No

Question: {input}
Are followup questions needed here:{agent_scratchpad}


> Entering new AgentExecutor chain...
Could not parse output: Yes, follow-up questions could be helpful to provide more detailed information about the provinces or to clarify specific aspects of Chinese provinces.

Follow-up questions might include:
- How many provinces are there in China?
- Are there any special administrative regions in China that are not provinces?
- Can you name some of the most populous provinces in China?
- What are the differences between a province and other administrative divisions in China?
- Are there any provinces that have unique cultural or historical significance in China?
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Intermediate answer:
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Apologies for the confusion. Let's address the questions one by one:

Question: 中国有哪些省份呢?
Intermediate answer:
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Certainly, here's the information you're seeking:

Question: 中国有哪些省份呢?

Final answer:
China is divided into 34 administrative divisions
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Final answer:
China is divided
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Final answer:
China is administratively divided into 23 provinces, 5 autonomous regions, 4 direct-controlled municipalities (Beijing, Tianjin, Shanghai, and Chongqing), and 2 special administrative regions (Hong Kong and Macau). Here are the provinces:

1. Hebei Province
2. Shanxi Province
3. Inner Mongolia Autonomous Region
4. Liaoning Province
5. Jilin Province
6. Heilongjiang Province
7. Shanghai Municipality
8. Jiangsu Province
9. Zhejiang Province
10. Anhui Province
11. Fujian Province
12. Jiangxi Province
13. Shandong Province
14. Henan Province
15. Hubei Province
16. Hunan Province
17. Guangdong Province
18. Guangxi Zhuang Autonomous Region
19. Hainan Province
20. Chongqing Municipality
21. Sichuan Province
22. Guizhou Province
23. Yunnan Province
24. Shaanxi Province
25. Gansu Province
26. Qinghai Province
27. Ningxia Hui Autonomous Region
28. Xinjiang Uyghur Autonomous Region

Please note that China also has a number of prefecture-level divisions, county-level divisions, and town-level divisions, but these are not considered provinces. If you have any specific follow-up questions or need further information, feel free to ask!
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseCould not parse output: Final answer:
China is divided into
For troubleshooting, visit: https://python.langchain.com/docs/troubleshooting/errors/OUTPUT_PARSING_FAILURE Invalid or incomplete responseTraceback (most recent call last):
  File "E:\workspace\gptLearning\gptLearning\ls10\agent\ls24_create_self_ask_with_search_agent.py", line 35, in <module>
    res = agent_executor.invoke({"input": "中国有哪些省份呢?"})

把问题改为：世界上最漂亮的人是谁， gpt3.5也宕机了，吐血

res = agent_executor.invoke({"input": "世界上最漂亮的人是谁?"})
print('res:',res)

gpt3.5执行失败：

gpt4执行成功：