星火认知大模型对话：（以spark 4.0 ultra 为例）
demo上的功能比较简陋，网络上搜到的比较残缺，很多功能缺失，我这里自己收集资料和运用编程知识做了整理，得到了自己想要的一些功能，比如持久化处理、对话历史记录持久化，自定义提示词传入等等。
希望能帮助到你们。
提前准备：
maven依赖：
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>4.10.0</version>
</dependency>
<dependency>
<groupId>com.squareup.okio</groupId>
<artifactId>okio</artifactId>
<version>2.10.0</version>
</dependency>配置类：
@EnableConfigurationProperties
@Data
@Component
@ConfigurationProperties("xf.config")
public class XFConfig {
private String appId;
private String apiSecret;
private String apiKey;
private String hostUrl;
private Integer maxResponseTime;
}配置信息放在yaml中：
xf:
config:
hostUrl: https://spark-api.xf-yun.com/v4.0/chat
appId: # 填写你的控制台的
apiSecret: #填写你的控制台的
apiKey: #填写你的控制台的
maxResponseTime: 40listener目录：
webClient：用于连接星火服务，无需修改，直接复制粘贴到此目录下即可
@Slf4j
@Component
public class XFWebClient {
@Autowired
private XFConfig xfConfig;
//线程池
private final ExecutorService makePool = Executors.newCachedThreadPool();
public XFWebClient(XFConfig xfConfig) {
this.xfConfig = xfConfig;
}
//通过webflux实现websocket长连接,异步流式响应,过滤掉除content的其他信息(可以控制响应速度)
public Flux<String> send(List<RoleContent> messages) throws Exception {
String authUrl = getAuthUrl(xfConfig.getHostUrl(), xfConfig.getApiKey(), xfConfig.getApiSecret());
String url = authUrl.replace("http://", "ws://").replace("https://", "wss://");
//获取连接器
ReactorNettyWebSocketClient client = new ReactorNettyWebSocketClient();
String json = JSON.toJSONString(createRequestParams(xfConfig.getAppId(), messages,null));
System.out.println("Request JSON: " + json); //检验参数
return Flux.create(fluxSink ->
makePool.execute(()-> client.execute(URI.create(url), session -> session //开启异步线程
.send(Flux.just(session.textMessage( //发送消息
Objects.requireNonNull(JSON.toJSONString(createRequestParams(xfConfig.getAppId(), messages,null)))
)))
.thenMany(session.receive()
.map(WebSocketMessage ->{
XfResponse response = JSON.parseObject(WebSocketMessage.getPayloadAsText(), XfResponse.class);
if(response.getHeader().getStatus() == 2){ //返回的是最后一个片段关闭
fluxSink.complete();
}
//处理返回的数据
String content = response.getPayload().getChoices().getText().get(0).getContent().replace("*","");
return content;
})
.doOnNext(fluxSink::next))
.then())
.block(Duration.ofSeconds(120))));//阻塞线程120s
}
/**
* @description: 发送请求至大模型方法
* @author: ChengLiang
* @date: 2023/10/19 16:27
* @param: [用户id, 请求内容, 返回结果监听器listener]
* @return: okhttp3.WebSocket
**/
public WebSocket sendMsg(String uid, List<RoleContent> questions, WebSocketListener listener) {
// 获取鉴权url
String authUrl = null;
try {
authUrl = getAuthUrl(xfConfig.getHostUrl(), xfConfig.getApiKey(), xfConfig.getApiSecret());
} catch (Exception e) {
log.error("鉴权失败：{}", e);
return null;
}
// 鉴权方法生成失败，直接返回 null
OkHttpClient okHttpClient = new OkHttpClient.Builder().build();
// 将 https/http 连接替换为 ws/wss 连接
String url = authUrl.replace("http://", "ws://").replace("https://", "wss://");
Request request = new Request.Builder().url(url).build();
// 建立 wss 连接
WebSocket webSocket = okHttpClient.newWebSocket(request, listener);
// 组装请求参数
JSONObject requestDTO = createRequestParams(uid, questions);
// 发送请求
webSocket.send(JSONObject.toJSONString(requestDTO));
return webSocket;
}
/**
* @description: 鉴权方法
* @author: ChengLiang
* @date: 2023/10/19 16:25
* @param: [讯飞大模型请求地址, apiKey, apiSecret]
* @return: java.lang.String
**/
public static String getAuthUrl(String hostUrl, String apiKey, String apiSecret) throws Exception {
URL url = new URL(hostUrl);
// 时间
SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US);
format.setTimeZone(TimeZone.getTimeZone("GMT"));
String date = format.format(new Date());
// 拼接
String preStr = "host: " + url.getHost() + "\n" +
"date: " + date + "\n" +
"GET " + url.getPath() + " HTTP/1.1";
// SHA256加密
Mac mac = Mac.getInstance("hmacsha256");
SecretKeySpec spec = new SecretKeySpec(apiSecret.getBytes(StandardCharsets.UTF_8), "hmacsha256");
mac.init(spec);
byte[] hexDigits = mac.doFinal(preStr.getBytes(StandardCharsets.UTF_8));
// Base64加密
String sha = Base64.getEncoder().encodeToString(hexDigits);
// 拼接
String authorization = String.format("api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"", apiKey, "hmac-sha256", "host date request-line", sha);
// 拼接地址
HttpUrl httpUrl = Objects.requireNonNull(HttpUrl.parse("https://" + url.getHost() + url.getPath())).newBuilder().//
addQueryParameter("authorization", Base64.getEncoder().encodeToString(authorization.getBytes(StandardCharsets.UTF_8))).//
addQueryParameter("date", date).//
addQueryParameter("host", url.getHost()).//
build();
return httpUrl.toString();
}
/**
* @description: 请求参数组装方法
* @author: ChengLiang
* @date: 2023/10/19 16:26
* @param: [用户id, 请求内容]
* @return: com.alibaba.fastjson.JSONObject
**/
public JSONObject createRequestParams(String uid, List<RoleContent> questions,String functions) {
JSONObject requestJson = new JSONObject();
// header参数
JSONObject header = new JSONObject();
header.put("app_id", xfConfig.getAppId());
header.put("uid", uid);
// parameter参数
JSONObject parameter = new JSONObject();
JSONObject chat = new JSONObject();
chat.put("domain", "4.0Ultra");
chat.put("temperature", 0.5);
chat.put("max_tokens", 4096);
parameter.put("chat", chat);
// payload参数
JSONObject payload = new JSONObject();
JSONObject message = new JSONObject();
JSONArray jsonArray = new JSONArray();
jsonArray.addAll(questions);
message.put("text", jsonArray);
payload.put("message", message);
payload.put("functions",functions);
requestJson.put("header", header);
requestJson.put("parameter", parameter);
requestJson.put("payload", payload);
return requestJson;
}
public JSONObject createRequestParams(String uid, List<RoleContent> questions) {
JSONObject requestJson = new JSONObject();
// header参数
JSONObject header = new JSONObject();
header.put("app_id", xfConfig.getAppId());
header.put("uid", uid);
// parameter参数
JSONObject parameter = new JSONObject();
JSONObject chat = new JSONObject();
chat.put("domain", "4.0Ultra");
chat.put("temperature", 0.5);
chat.put("max_tokens", 4096);
parameter.put("chat", chat);
// payload参数
JSONObject payload = new JSONObject();
JSONObject message = new JSONObject();
JSONArray jsonArray = new JSONArray();
jsonArray.addAll(questions);
message.put("text", jsonArray);
payload.put("message", message);
requestJson.put("header", header);
requestJson.put("parameter", parameter);
requestJson.put("payload", payload);
return requestJson;
}
}socketListener：用于监听和星火认知大模型的socket连接，也是直接复制粘贴到listener目录下：
可以在todo处进行持久化处理，我这里不在这里处理，下面有其他代码我会专门做处理。
@Slf4j
public class XFWebSocketListener extends WebSocketListener {
//断开websocket标志位
private boolean wsCloseFlag = false;
//语句组装buffer，将大模型返回结果全部接收，在组装成一句话返回
private StringBuilder answer = new StringBuilder();
public String getAnswer() {
return answer.toString();
}
public boolean isWsCloseFlag() {
return wsCloseFlag;
}
@Override
public void onOpen(WebSocket webSocket, Response response) {
super.onOpen(webSocket, response);
log.info("大模型服务器连接成功！");
}
@Override
public void onMessage(WebSocket webSocket, String text) {
super.onMessage(webSocket, text);
JsonParse myJsonParse = JSON.parseObject(text, JsonParse.class);
log.info("myJsonParse:{}", JSON.toJSONString(myJsonParse));
if (myJsonParse.getHeader().getCode() != 0) {
log.error("发生错误，错误信息为:{}", JSON.toJSONString(myJsonParse.getHeader()));
this.answer.append("大模型响应异常，请联系管理员");
// 关闭连接标识
wsCloseFlag = true;
return;
}
List<Text> textList = myJsonParse.getPayload().getChoices().getText();
for (Text temp : textList) {
log.info("返回结果信息为：【{}】", JSON.toJSONString(temp));
this.answer.append(temp.getContent());
}
log.info("result:{}", this.answer.toString());
if (myJsonParse.getHeader().getStatus() == 2) {
wsCloseFlag = true;
//todo 将问答信息入库进行记录，可自行实现
}
}
@Override
public void onFailure(WebSocket webSocket, Throwable t, Response response) {
super.onFailure(webSocket, t, response);
try {
if (null != response) {
int code = response.code();
log.error("onFailure body:{}", response.body().string());
if (101 != code) {
log.error("讯飞星火大模型连接异常");
}
}
} catch (IOException e) {
log.error("IO异常：{}", e);
}
}
}实体类：
有点多，要慢慢复制粘贴。
@Data
public class Choices {
private List<Text> text;
}@lombok.Data
public class Data {
int status;
String audio;
}这个FileDetail实体类用于讯飞多模态分析的表情识别：
@Data
public class FileDetail {
private int code;
private String fileName;
private int label;
private List<Object> labels;
private String name;
private double rate; // 如果rate是字符串，请改为String
private List<Double> rates;
private boolean review;
private List<Object> subLabels;
private String tag;
}@Data
public class Header {
private int code;
private int status;
private String sid;
}@Data
public class JsonParse {
private Header header;
private Payload payload;
}public class NettyGroup {
private static ChannelGroup channelGroup = new DefaultChannelGroup(GlobalEventExecutor.INSTANCE);
/**
* 存放用户与Chanel的对应信息，用于给指定用户发送消息
*/
private static ConcurrentHashMap<String, Channel> channelMap = new ConcurrentHashMap<>();
private NettyGroup() {
}
/**
* 获取channel组
*/
public static ChannelGroup getChannelGroup() {
return channelGroup;
}
/**
* 获取连接channel map
*/
public static ConcurrentHashMap<String, Channel> getUserChannelMap() {
return channelMap;
}
}@Data
public class Payload {
private Choices choices;
}@Getter
@Setter
@ToString(callSuper = true)
@AllArgsConstructor
@NoArgsConstructor
public class ResultBean<T> {
private String errorCode;
private String message;
private T data;
public ResultBean(T data) {
this.errorCode = ErrorMessage.SUCCESS.getErrorCode();
this.message = ErrorMessage.SUCCESS.getMessage();
this.data = data;
}
public ResultBean(ErrorMessage errorMessage, T data) {
this.errorCode = errorMessage.getErrorCode();
this.message = errorMessage.getMessage();
this.data = data;
}
public static <T> ResultBean success(T data) {
ResultBean resultBean = new ResultBean(data);
return resultBean;
}
public static <T> ResultBean fail(T data) {
ResultBean resultBean = new ResultBean(ErrorMessage.FAIL.getErrorCode(), ErrorMessage.FAIL.getMessage(), data);
return resultBean;
}
public enum ErrorMessage {
SUCCESS("0", "success"),
FAIL("001", "fail"),
NOAUTH("1001", "非法访问");
private String errorCode;
private String message;
ErrorMessage(String errorCode, String message) {
this.errorCode = errorCode;
this.message = message;
}
public String getErrorCode() {
return errorCode;
}
public void setErrorCode(String errorCode) {
this.errorCode = errorCode;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
}
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class RoleContent {
public static final String ROLE_USER = "user";
public static final String ROLE_ASSISTANT = "assistant";
private String role;
private String content;
public static RoleContent createUserRoleContent(String content) {
return new RoleContent(ROLE_USER, content);
}
public static RoleContent createAssistantRoleContent(String content) {
return new RoleContent(ROLE_ASSISTANT, content);
}
}这个类很重要，记住它：
@Data
@NoArgsConstructor
@TableName("t_role_content")
public class RoleContentPo implements Serializable {
private static final long serialVersionUID = 1666L;
@TableId(type = IdType.AUTO)
private Long id;
private String uid;
private String role; //user,
private String content;
public RoleContentPo(String uid, String role, String content){
this.uid = uid;
this.role = role;
this.content = content;
}
private Date sendTime;
private Long scid;
private Long rcid;
private String type;
private Long voiceLength;
@TableLogic
//是否删除
private Integer isDeleted;
}@Data
public class Text {
private String role;
private String content;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class XFInputQuestion {
private String uid;
private String context;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class XFInputQuestionFunctionCall {
private String uid;
private String context;
private Object functionsParams;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class Choices {
private int status;
private int seq;
private List<Message> text;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class ResponseHeader {
private Integer code;
private String message;
private String sid;
private Integer status;
}@Data
public class ResponsePayload implements Serializable {
private Choices choices;
private Usage usage;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class TextUsage implements Serializable {
@JsonProperty("prompt_tokens")
private Integer promptTokens;
@JsonProperty("completion_tokens")
private Integer completionTokens;
@JsonProperty("total_tokens")
private Integer totalTokens;
@JsonProperty("question_tokens")
private Integer questionTokens;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class Usage implements Serializable {
private TextUsage text;
}@AllArgsConstructor
@NoArgsConstructor
@Data
public class XfResponse implements Serializable {
private ResponseHeader header;
private ResponsePayload payload;
}还可以再建一个history子目录，用于存放有关历史对话的实体类：
@Data
public class AiHistoryPage {
//发送者的id
private String scid;
//当前页码
private Integer pageNum;
//每一页的数据量大小
private Integer pageSize;
//最大页数
private Integer pageMaxCount;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class AiHistoryRecords {
private Integer recordsNum;
private ArrayList<HistoryInfo> history;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class HistoryDataVo {
private String uid;
private Integer userId;
@JsonFormat(pattern = "yyyy-MM-dd", timezone = "GMT+8")
private Date uTime;
private String lastQuestion;
}@Data
@AllArgsConstructor
@NoArgsConstructor
public class HistoryInfo {
@TableId
private Integer id;
private String uid;
private String scid;
private String LastMsgTime;
private String LastQuestionMsg;
}mysql表：只有两个：
CREATE TABLE `t_role_content` (
`id` int unsigned NOT NULL AUTO_INCREMENT,
`uid` varchar(50) NOT NULL DEFAULT 'ergou' COMMENT '对话id',
`role` varchar(50) NOT NULL DEFAULT 'user' COMMENT '对话角色',
`content` text NOT NULL COMMENT '对话内容',
`is_deleted` tinyint unsigned NOT NULL DEFAULT '0' COMMENT '删除标志，默认0不删除，1删除',
`send_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`scid` bigint NOT NULL DEFAULT '1773322941155852290' COMMENT '发送人',
`rcid` bigint NOT NULL DEFAULT '12345678' COMMENT '接收人',
`type` varchar(50) NOT NULL DEFAULT 'text' COMMENT '消息类型',
`read` int DEFAULT '0' COMMENT '是否已读',
`voice_length` int DEFAULT '0' COMMENT '语音长度',
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2056 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci ROW_FORMAT=DYNAMIC COMMENT='ai对话历史记录';CREATE TABLE `history_info` (
`id` bigint NOT NULL AUTO_INCREMENT COMMENT '主键',
`scid` char(11) DEFAULT NULL COMMENT '电话',
`uid` varchar(50) DEFAULT NULL COMMENT '对话uid',
`last_msg_time` varchar(50) DEFAULT '某个时间' COMMENT '最后一条消息的时间',
`last_question_msg` text COMMENT '最后一条消息的内容',
`is_deleted` tinyint NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `uid` (`uid`)
) ENGINE=InnoDB AUTO_INCREMENT=2147119107 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci ROW_FORMAT=DYNAMIC COMMENT='对话历史记录';接着建立mapper：
@Mapper
public interface HistoryInfoMapper extends BaseMapper<HistoryInfo> {
}@Mapper
public interface RoleContentPoMapper extends BaseMapper<RoleContentPo> {
}service以及其impl：
public interface AiHistoryRecordsService{
ResultData<AiHistoryRecords> getAiHistory(AiHistoryPage aiHistoryPage);
}public interface HistoryInfoService extends IService<HistoryInfo> {
}public interface PushService {
void pushToOne(String uid, String text);
void pushToAll(String text);
//测试账号只有2个并发，此处只使用一个，若是生产环境允许多个并发，可以采用分布式锁
ResultData pushMessageToXFServer(RoleContentPo roleContentPo, String background);
ResultData<RoleContentPo> pushVoiceMessageToXFServer(RoleContentPo roleContentPo, String background,String voiceMessage);
}public interface RoleContentPoService extends IService<RoleContentPo> {
ResultData getAllMessage(AiMessagePage page);
ResultData getAllMsgPageMax(AiMessagePage page);
ResultData<RoleContentPo> voiceMsgToAi(RoleContentPo roleContentPo, String message);
}impl：
@Service
public class AiHistoryRecordsServiceImpl implements AiHistoryRecordsService {
@Resource
private HistoryInfoMapper historyInfoMapper;
@Override
public ResultData<AiHistoryRecords> getAiHistory(AiHistoryPage aiHistoryPage) {
//分页构造器
IPage<HistoryInfo> page = new Page<>(aiHistoryPage.getPageNum(),aiHistoryPage.getPageSize());
LambdaQueryWrapper<HistoryInfo> lambdaQueryWrapper = new LambdaQueryWrapper<>();
lambdaQueryWrapper.eq(HistoryInfo::getScid,aiHistoryPage.getScid());
//将分页信息查询出来
IPage<HistoryInfo> selectPage = historyInfoMapper.selectPage(page, lambdaQueryWrapper);
List<HistoryInfo> records = selectPage.getRecords();
ListUtil.reverse(records);
ArrayList<HistoryInfo> historyInfos = ListUtil.toList(records);
AiHistoryRecords aiHistoryRecords = new AiHistoryRecords();
aiHistoryRecords.setHistory(historyInfos);
aiHistoryRecords.setRecordsNum(records.size());
return ResultData.success(aiHistoryRecords);
}
}@Service
public class HistoryInfoServiceImpl extends ServiceImpl<HistoryInfoMapper, HistoryInfo> implements HistoryInfoService {
}之后主要调用的就是这个PushService的pushMessageToXFServer方法，只需要传入用户输入信息和提示词，就可以获得ai输出信息。
如果想要前端实时接收流式数据，要在todo处，进行和前端的webSocket连接。
@Slf4j
@Service
public class PushServiceImpl implements PushService {
@Autowired
private XFConfig xfConfig;
@Autowired
private XFWebClient xfWebClient;
@Resource
private RoleContentPoMapper roleContentPoMapper;
@Resource
private WebSocketServer webSocketServer;
@Override
public void pushToOne(String uid, String text) {
if (StringUtils.isEmpty(uid) || StringUtils.isEmpty(text)) {
log.error("uid或text均不能为空");
throw new RuntimeException("uid或text均不能为空");
}
ConcurrentHashMap<String, Channel> userChannelMap = NettyGroup.getUserChannelMap();
for (String channelId : userChannelMap.keySet()) {
if (channelId.equals(uid)) {
Channel channel = userChannelMap.get(channelId);
if (channel != null) {
ResultBean success = ResultBean.success(text);
channel.writeAndFlush(new TextWebSocketFrame(JSON.toJSONString(success)));
log.info("信息发送成功：{}", JSON.toJSONString(success));
} else {
log.error("该id对于channelId不存在！");
}
return;
}
}
log.error("该用户不存在！");
}
@Override
public void pushToAll(String text) {
String trim = text.trim();
ResultBean success = ResultBean.success(trim);
NettyGroup.getChannelGroup().writeAndFlush(new TextWebSocketFrame(JSON.toJSONString(success)));
log.info("信息推送成功：{}", JSON.toJSONString(success));
}
//测试账号只有2个并发，此处只使用一个，若是生产环境允许多个并发，可以采用分布式锁
// @Async
@Override
public synchronized ResultData<RoleContentPo> pushMessageToXFServer(RoleContentPo roleContentPo, String background) {
//****************** part1 ************************************************
//从数据库中查询历史记录
LambdaQueryWrapper<RoleContentPo> lambdaQueryWrapper = new LambdaQueryWrapper<>();
lambdaQueryWrapper.eq(RoleContentPo::getUid,roleContentPo.getUid()); //查询对应uid的对话历史记录
lambdaQueryWrapper.orderByAsc(RoleContentPo::getSendTime); //要根据创建时间升序排序
List<RoleContentPo> historyPoList = roleContentPoMapper.selectList(lambdaQueryWrapper);
//判断是否历史记录的消息字数是否大于限制
if(!canAddHistory(historyPoList) && historyPoList != null){
beforeAddHistory(historyPoList); //如果字数太多，对历史记录进行清理
}
List<RoleContent> historyList = null;
if(historyPoList != null){
//RoleContentPo是用来接收数据库的数据的，ai要使用的是RoleContent类型的数据，所以要使用stream流将类型转换一下。
historyList = historyPoList.stream().map((item) -> {
RoleContent roleContent = new RoleContent();
roleContent.setRole(item.getRole());
roleContent.setContent(item.getContent());
return roleContent;
}).collect(Collectors.toList());
}else {
historyList = new ArrayList<>(); //如果数据库中没有数据，就用一个空的历史记录给ai
}
ArrayList<RoleContent> questions = new ArrayList<>();
//************************************** part2 ************************************
//设置对话背景
RoleContent system = new RoleContent();
system.setRole("system");
system.setContent(background);
questions.add(system);//添加对话背景到要给ai的集合中
//将历史消息加入其中
if (historyList.size() > 0){
questions.addAll(historyList);
}
//接收用户的输入信息
RoleContent userRoleContent = RoleContent.createUserRoleContent(roleContentPo.getContent());
//将用户的输入信息加入到要给ai的集合中
questions.add(userRoleContent);
//将用户的消息输入也加入到数据库
RoleContentPo newUserRoleContent = new RoleContentPo();
newUserRoleContent.setContent(userRoleContent.getContent());
newUserRoleContent.setRole(userRoleContent.getRole());
newUserRoleContent.setUid(roleContentPo.getUid());
newUserRoleContent.setSendTime(new Date(System.currentTimeMillis()));//将当前时间存入数据库
newUserRoleContent.setScid(roleContentPo.getScid());
roleContentPoMapper.insert(newUserRoleContent);
//将信息给ai，让ai处理后，结果会给到xfWebSocketListener上
XFWebSocketListener xfWebSocketListener = new XFWebSocketListener();
WebSocket webSocket = xfWebClient.sendMsg(roleContentPo.getUid(), questions, xfWebSocketListener);
//************************************* part3 *****************************************
if (webSocket == null) {
log.error("webSocket连接异常");
ResultBean.fail("请求异常，请联系管理员");
}
try {
int count = 0;
int maxCount = xfConfig.getMaxResponseTime() * 5;
while (count <= maxCount) {
Thread.sleep(2000);
log.info("大模型的回答是："+xfWebSocketListener.getAnswer());
//这个地方是每两秒可以获得ai进行途中未完成的回答，可以进行异步返回给前端
//todo: 在此处实现异步返回给前端（待实现）
webSocketServer.sendToOne(roleContentPo.getScid().toString(),xfWebSocketListener.getAnswer());
log.info("###############################################################################\n################################################################################################");
if (xfWebSocketListener.isWsCloseFlag()) {
break;
}
count++;
}
if (count > maxCount) {
return ResultData.fail(ReturnCodeEnum.RC500.getCode(),"响应超时，请联系相关人员");
}
//获取ai的回答的总内容
String totalAnswer = xfWebSocketListener.getAnswer();
//ai的回答也要存进数据库中
RoleContentPo roleContent = new RoleContentPo();
roleContent.setRole("assistant");
roleContent.setContent(totalAnswer);
roleContent.setUid(roleContentPo.getUid());
roleContent.setSendTime(new Date(System.currentTimeMillis()));
roleContent.setScid(roleContentPo.getRcid());
roleContent.setRcid(roleContentPo.getScid());
roleContent.setType("text");
roleContentPoMapper.insert(roleContent);
return ResultData.success(roleContent);
} catch (Exception e) {
log.error("请求异常：{}", e);
} finally {
webSocket.close(1000, "");
}
return ResultData.success(null);
}
@Override
public synchronized ResultData<RoleContentPo> pushVoiceMessageToXFServer(RoleContentPo roleContentPo, String background,String voiceMessage) {
//****************** part1 ************************************************
//从数据库中查询历史记录
LambdaQueryWrapper<RoleContentPo> lambdaQueryWrapper = new LambdaQueryWrapper<>();
lambdaQueryWrapper.eq(RoleContentPo::getUid,roleContentPo.getUid()); //查询对应uid的对话历史记录
lambdaQueryWrapper.orderByAsc(RoleContentPo::getSendTime); //要根据创建时间升序排序
List<RoleContentPo> historyPoList = roleContentPoMapper.selectList(lambdaQueryWrapper);
//判断是否历史记录的消息字数是否大于限制
if(!canAddHistory(historyPoList) && historyPoList != null){
beforeAddHistory(historyPoList); //如果字数太多，对历史记录进行清理
}
List<RoleContent> historyList = null;
if(historyPoList != null){
//RoleContentPo是用来接收数据库的数据的，ai要使用的是RoleContent类型的数据，所以要使用stream流将类型转换一下。
historyList = historyPoList.stream().map((item) -> {
RoleContent roleContent = new RoleContent();
roleContent.setRole(item.getRole());
roleContent.setContent(item.getContent());
return roleContent;
}).collect(Collectors.toList());
}else {
historyList = new ArrayList<>(); //如果数据库中没有数据，就用一个空的历史记录给ai
}
ArrayList<RoleContent> questions = new ArrayList<>();
//************************************** part2 ************************************
//设置对话背景
RoleContent system = new RoleContent();
system.setRole("system");
system.setContent(background);
questions.add(system);//添加对话背景到要给ai的集合中
//将历史消息加入其中
if (historyList.size() > 0){
questions.addAll(historyList);
}
//接收用户的输入信息
RoleContent userRoleContent = RoleContent.createUserRoleContent(voiceMessage);
//将用户的输入信息加入到要给ai的集合中
questions.add(userRoleContent);
//将信息给ai，让ai处理后，结果会给到xfWebSocketListener上
XFWebSocketListener xfWebSocketListener = new XFWebSocketListener();
WebSocket webSocket = xfWebClient.sendMsg(roleContentPo.getUid(), questions, xfWebSocketListener);
//************************************* part3 *****************************************
if (webSocket == null) {
log.error("webSocket连接异常");
ResultBean.fail("请求异常，请联系管理员");
}
try {
int count = 0;
int maxCount = xfConfig.getMaxResponseTime() * 5;
while (count <= maxCount) {
Thread.sleep(2000);
log.info("大模型的回答是："+xfWebSocketListener.getAnswer());
//这个地方是每两秒可以获得ai进行途中未完成的回答，可以进行异步返回给前端
//todo: 在此处实现异步返回给前端（待实现）
log.info("###############################################################################\n################################################################################################");
if (xfWebSocketListener.isWsCloseFlag()) {
break;
}
count++;
}
if (count > maxCount) {
return ResultData.fail(ReturnCodeEnum.RC500.getCode(),"响应超时，请联系相关人员");
}
//获取ai的回答的总内容
String totalAnswer = xfWebSocketListener.getAnswer();
//ai的回答也要存进数据库中
RoleContentPo roleContent = new RoleContentPo();
roleContent.setRole("assistant");
roleContent.setContent(totalAnswer);
roleContent.setUid(roleContent.getUid());
roleContent.setSendTime(new Date(System.currentTimeMillis()));
roleContent.setScid(roleContentPo.getRcid());
roleContent.setRcid(roleContent.getScid());
roleContent.setType("text");
roleContentPoMapper.insert(roleContent);
return ResultData.success(roleContent);
} catch (Exception e) {
log.error("请求异常：{}", e);
} finally {
webSocket.close(1000, "");
}
return ResultData.success(null);
}
public void beforeAddHistory(List<RoleContentPo> historyList){ // 如果历史记录过长，就删除5条记录
for (int i = 0;i < 5;i++) {
RoleContentPo roleContentPo = historyList.get(i);
roleContentPoMapper.deleteById(roleContentPo);//数据库中的历史记录删除5条
historyList.remove(i);//给ai的信息中也删除5条
}
}
public boolean canAddHistory(List<RoleContentPo> historyList){ // 由于历史记录最大上线1.2W左右，需要判断是能能加入历史
int history_length=0;
for(RoleContentPo temp:historyList){
history_length=history_length+temp.getContent().length();
}
if(history_length>11000){
return false;
}else{
return true;
}
}
}@Service
public class RoleContentPoServiceImpl extends ServiceImpl<RoleContentPoMapper, RoleContentPo> implements RoleContentPoService {
@Resource
private RoleContentPoMapper roleContentPoMapper;
@Resource
private PushService pushService;
@Override
public ResultData<List<RoleContentPo>> getAllMessage(AiMessagePage page) {
//构造分页构造器
IPage<RoleContentPo> optionPage = new Page<>(page.getPageNum(),page.getPageSize());
//构造条件构造器
LambdaQueryWrapper<RoleContentPo> queryWrapper = new LambdaQueryWrapper<>();
//当uid等于指定的值，将其记录查询出来
queryWrapper.eq(RoleContentPo::getUid,page.getUid());
//根据查询时间升序排列
queryWrapper.orderByDesc(RoleContentPo::getSendTime);
//分页查找
roleContentPoMapper.selectPage(optionPage,queryWrapper);
//查找的结果
List<RoleContentPo> records = optionPage.getRecords();
return ResultData.success(records);
}
@Override
public ResultData<Long> getAllMsgPageMax(AiMessagePage page) {
//构造分页构造器
IPage<RoleContentPo> optionPage = new Page<>(page.getPageNum(),page.getPageSize());
//构造条件构造器
LambdaQueryWrapper<RoleContentPo> queryWrapper = new LambdaQueryWrapper<>();
//当uid等于指定的值，将其记录查询出来
queryWrapper.eq(RoleContentPo::getUid,page.getUid());
//根据查询时间升序排列
queryWrapper.orderByAsc(RoleContentPo::getSendTime);
//分页查找
roleContentPoMapper.selectPage(optionPage,queryWrapper);
return ResultData.success(optionPage.getPages());
}
@Override
public ResultData<RoleContentPo> voiceMsgToAi(RoleContentPo roleContentPo,String voiceMessage) {
return pushService.pushVoiceMessageToXFServer(roleContentPo,"",voiceMessage);
}
}使用案例：
String answer = userAnswer.getAnswer()[0];
RoleContentPo roleContentPo = new RoleContentPo();
roleContentPo.setUid(userExamVo.getExamId());
String content = """
问题：""" + question.getText() + """
参考答案：""" + question.getAnswer()[0] + """
答案：""" + answer + """
满分：""" + question.getScore() + """
""";
roleContentPo.setContent(content);
roleContentPo.setRole("user");
roleContentPo.setType("text");
roleContentPo.setRcid(12345678L);
roleContentPo.setScid(Long.parseLong(userExamVo.getUserId()));
String background = """
给你一个填空题的题目和答案，再告诉你这题的满分是多少，你只需要返回一个分数的数字，不要出现任何其他内容，我只要这个数字，不要出现任何其他内容。
关于评分规则，如果一道比较灵活的题目，有多种作答，或者应该要分多个点，或者需要更多元化的答案，那么请分析用户的回答的质量来评分。
参考答案只是一个用于参考的答案，不一定是唯一的正确答案，你可以根据你的判断来评分。
你要分析的信息的格式为：
问题：{问题内容}
参考答案：{参考答案}
答案：{用户的回答}
满分：{满分}
注意，请严格按照格式要求返回分数，不要出现任何其他内容，我只要这个数字，不要出现任何其他内容。
""";
ResultData resultData = pushService.pushMessageToXFServer(roleContentPo, background);
RoleContentPo data1 = (RoleContentPo) resultData.getData();
String data = data1.getContent();
int resultScore = Integer.parseInt(data);
score += resultScore;
userAnswer.setScore(resultScore);语音合成服务TTS：
oss工具类
这里建议使用阿里的Oss对象云存储来存取文件信息：
这里我还多做了一个可以传入File类的对象的存取，在TTS中会使用到
@Component
public class OssUtils {
//读取配置文件的内容
@Value("${aliyun.oss.file.endpoint}")
private String ENDPOINT;
@Value("${aliyun.oss.file.access-key-id}")
private String ACCESS_KEY_ID;
@Value("${aliyun.oss.file.serct-access-key}")
private String ACCESS_KEY_SECRET;
@Value("${aliyun.oss.file.bucket}")
private String bucketName;
private OSS ossClient;
//初始化oss服务
public void init() {
ossClient = new OSSClientBuilder().build(ENDPOINT, ACCESS_KEY_ID, ACCESS_KEY_SECRET);
}
//上传文件
// file:文件
public String uploadFile(MultipartFile file) throws IOException {
//获取文件的输入流
InputStream inputstream = file.getInputStream();
String filename = file.getOriginalFilename();
//保证文件唯一性
String uuid = UUID.randomUUID().toString().replaceAll("-", "");
filename = uuid + filename;
//按照类别进行分类
// 判断文件类型
String contentType = file.getContentType();
String fileType = "unknown";
if (contentType != null) {
if (contentType.startsWith("image/")) {
fileType = "image"; //图像文件
} else if (contentType.startsWith("application/")) {
fileType = "application"; //应用程序文件
} else if (contentType.startsWith("text/")) {
fileType = "text"; //文本文件 (包括 HTML、CSS、JavaScript)
} else if (contentType.startsWith("video/")) {
fileType = "video"; //视频文件
} else if (contentType.startsWith("audio/")) {
fileType = "audio"; //音频文件
}
} else {
fileType = "other"; //其他文件
}
//文件路劲
filename = fileType + "/" + filename;
try {
ossClient.putObject(bucketName, filename, inputstream);
// // 设置 URL 过期时间
// Date expiration = new Date(System.currentTimeMillis() + 60 * 1000 * 60);
// URL url = ossClient.generatePresignedUrl(bucketName, filename, expiration);
// return url.toString();
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
String url = "https://" + bucketName + "." + ENDPOINT + "/" + filename;
// return generatePresignedUrl(filename, 60 * 60);
return url;
}
public String uploadFile(File file,String type) throws IOException {
//获取文件的输入流
InputStream inputstream = new FileInputStream(file);
String filename = file.getName();
if (file.exists() && file.isFile()) {
long fileSizeInBytes = file.length();
System.out.println("文件的字节大小为: " + fileSizeInBytes + " 字节");
} else {
System.out.println("文件不存在或不是一个常规文件。");
}
//保证文件唯一性
String uuid = UUID.randomUUID().toString().replaceAll("-", "");
filename = uuid + filename;
//按照类别进行分类
// 判断文件类型
String contentType = type;
String fileType = "unknown";
if (contentType != null) {
if (contentType.startsWith("image/")) {
fileType = "image"; //图像文件
} else if (contentType.startsWith("application/")) {
fileType = "application"; //应用程序文件
} else if (contentType.startsWith("text/")) {
fileType = "text"; //文本文件 (包括 HTML、CSS、JavaScript)
} else if (contentType.startsWith("video/")) {
fileType = "video"; //视频文件
} else if (contentType.startsWith("audio/")) {
fileType = "audio"; //音频文件
}
} else {
fileType = "other"; //其他文件
}
//文件路劲
filename = fileType + "/" + filename;
try {
ossClient.putObject(bucketName, filename, inputstream);
// // 设置 URL 过期时间
// Date expiration = new Date(System.currentTimeMillis() + 60 * 1000 * 60);
// URL url = ossClient.generatePresignedUrl(bucketName, filename, expiration);
// return url.toString();
System.out.println("上传成功，文件名为：" + filename);
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
if (ossClient != null) {
ossClient.shutdown();
}
}
String url = "https://" + bucketName + "." + ENDPOINT + "/" + filename;
// return generatePresignedUrl(filename, 60 * 60);
return url;
}
//下载文件
// objectName:oss中的相对路径
// localPath:本地文件路径
public void downloadFile(String objectName, String localPath) throws Exception {
try {
//获取文件的名称
String fileName = new File(objectName).getName();
// 调用ossClient.getObject返回一个OSSObject实例，该实例包含文件内容及文件元数据。
// OSSObject ossObject = ossClient.getObject(bucketName, objectName);
// 调用ossObject.getObjectContent获取文件输入流，可读取此输入流获取其内容。
// InputStream content = ossObject.getObjectContent();
// 构建完整的文件路径
File path = new File(localPath, fileName);
// 检查并创建目录
File parentDir = path.getParentFile();
if (parentDir != null && !parentDir.exists() && !parentDir.mkdirs()) {
throw new IOException("无法创建目录: " + parentDir.getAbsolutePath());
}
// 检查文件是否可以创建
if (!path.exists() && !path.createNewFile()) {
throw new IOException("无法创建文件: " + path.getAbsolutePath());
}
ossClient.getObject(new GetObjectRequest(bucketName, objectName), path);
//流式下载
// if (content != null) {
// try (InputStream inputStream = content;
// OutputStream outputStream = new FileOutputStream(path)) {
// byte[] buffer = new byte[1024];
// int length;
// while ((length = inputStream.read(buffer)) > 0) {
// outputStream.write(buffer, 0, length);
// }
// }
// }
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
// if (ossClient != null) {
// ossClient.shutdown();
// }
}
}
//列举指定目录的所有的文件
/**
* @param folderPath:文件夹路径
* @return java.util.List<com.aliyun.oss.model.OSSObjectSummary>
* @author zhang
* @create 2024/10/31
**/
//OSSobjectSummary存储了元数据
public List<OSSObjectSummary> listAllObjects(String folderPath) {
List<OSSObjectSummary> objectSummaries = null;
try {
// 创建 ListObjectsRequest 对象并设置前缀
ListObjectsRequest listObjectsRequest = new ListObjectsRequest(bucketName);
listObjectsRequest.setPrefix(folderPath);
// ossClient.listObjects返回ObjectListing实例，包含此次listObject请求的返回结果。
ObjectListing objectListing = ossClient.listObjects(listObjectsRequest);
objectSummaries = objectListing.getObjectSummaries();
//文件对应的相对路劲打印出来
for (OSSObjectSummary objectSummary : objectListing.getObjectSummaries()) {
System.out.println(" - " + objectSummary.getKey() + " " +
"(size = " + objectSummary.getSize() + ")");
}
while (true) {
// 如果有下一批，继续获取
if (objectListing.isTruncated()) {
objectListing = ossClient.listObjects(String.valueOf(objectListing));
objectSummaries.addAll(objectListing.getObjectSummaries());
} else {
break;
}
}
return objectSummaries;
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
// if (ossClient != null) {
// ossClient.shutdown();
// }
}
return objectSummaries;
}
public List<String> getListUrl(String folderPath){
ArrayList<String> listUrl = new ArrayList<>();
List<OSSObjectSummary> summaries = listAllObjects(folderPath);
for (OSSObjectSummary summary:summaries){
String fileName = summary.getKey();
String url = "https://" + bucketName + "." + ENDPOINT + "/" +fileName;
System.out.println(url);
listUrl.add(url);
}
return listUrl;
}
//删除文件
//objectName:oss中的相对路径
public void deleteFile(String objectName) {
try {
// 删除文件。
ossClient.deleteObject(bucketName, objectName);
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
// if (ossClient != null) {
// ossClient.shutdown();
// }
}
}
//查看文件是否已经存在:默认不存在 没怎么必要,上传是写入了uuid唯一标识
/**
* @param objectName:文件的相对路径
* @return boolean
* @author xinggang
* @create 2024/10/31
**/
public boolean isExist(String objectName) {
boolean found = false;
try {
// 判断文件是否存在。如果返回值为true，则文件存在，否则存储空间或者文件不存在。
// 设置是否进行重定向或者镜像回源。默认值为true，表示忽略302重定向和镜像回源；如果设置isINoss为false，则进行302重定向或者镜像回源。
//boolean isINoss = true;
found = ossClient.doesObjectExist(bucketName, objectName);
//boolean found = ossClient.doesObjectExist(bucketName, objectName, isINoss);
return found;
} catch (OSSException oe) {
System.out.println("Caught an OSSException, which means your request made it to OSS, "
+ "but was rejected with an error response for some reason.");
System.out.println("Error Message:" + oe.getErrorMessage());
System.out.println("Error Code:" + oe.getErrorCode());
System.out.println("Request ID:" + oe.getRequestId());
System.out.println("Host ID:" + oe.getHostId());
} finally {
}
return found;
}
// 生成带签名的临时访问 URL,用于URL安全
// filename:oss中的相对路径
// expires:过期时间(分钟)
public String generatePresignedUrl(String filename, long expirationInSeconds) {
// 设置 URL 过期时间
Date expiration = new Date(System.currentTimeMillis() + expirationInSeconds * 1000);
// 生成带签名的 URL
GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucketName, filename);
request.setExpiration(expiration);
URL url = ossClient.generatePresignedUrl(request);
// 返回 URL 字符串
return url.toString();
}
public byte[] getObject(String pathUrl) {
//初始化
ossClient = new OSSClientBuilder().build(ENDPOINT, ACCESS_KEY_ID, ACCESS_KEY_SECRET);
String key = pathUrl.replace(ENDPOINT + "", "").replaceAll(bucketName+".","").replaceAll("https://","");
int index = key.indexOf("/");
// String bucketName = key.substring(0, index);
String filePath = key.substring(index + 1);
InputStream inputStream = null;
try {
// 获取文件输入流
inputStream = ossClient.getObject(bucketName, filePath).getObjectContent();
} catch (Exception e) {
e.printStackTrace();
}
// 读取文件内容
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[1024];
int len;
while (true) {
try {
if (!((len = inputStream.read(buffer)) != -1)) break;
} catch (IOException e) {
throw new RuntimeException(e);
}
outputStream.write(buffer, 0, len);
}
return outputStream.toByteArray();
}
//关闭oss服务
public void shutdown() {
if (ossClient != null) {
ossClient.shutdown();
}
}
}aliyun: #阿里云的配置
oss:
file:
endpoint: 。。。。。。。
access-key-id: 。。。。。。
serct-access-key: 。。。。。。。
bucket: 。。。。。。TTS工具类：
调用createVoice方法，传入想要合成的语音的内容，就会返回相应文件的oss的访问url。在websocket连接到星火服务之后，要sleep一小会，因为文件写出到指定文件需要一定时间。之前我没有sleep的时候，还没写出就给oss了，导致文件为0字节。所以这个sleep是有必要的，我这里有点久，可以适当短一点，能写出完成就行。
这里把它注册成了一个组件，在使用的时候注入，并且要设置其开启状态：案例：
@Resource
private WebTtsWs webTtsWs
@PostMapping("/createVoice")
public ResultData<String[]> createVoice(@RequestBody String voice) throws Exception {
webTtsWs.setWsCloseFlag(false);
String resultString = webTtsWs.createVoice(voice);
Thread.sleep(3000);//等语音生成完成了再关闭，给一点等待时间
webTtsWs.setWsCloseFlag(true);
return ResultData.success(resultString);
}@Service
public class WebTtsWs {
@Resource
private OssUtils ossUtils;
// 地址与鉴权信息
public static final String hostUrl = "https://tts-api.xfyun.cn/v2/tts";
// 均到控制台-语音合成页面获取
public static final String appid = "。。。。。";
public static final String apiSecret = "。。。。。。。";
public static final String apiKey = "。。。。。。。。";
// 合成文本
public static final String TEXT = "讯飞的文字合成语音功能，测试成功";
// 合成文本编码格式
public static final String TTE = "UTF8"; // 小语种必须使用UNICODE编码作为值
// 发音人参数。到控制台-我的应用-语音合成-添加试用或购买发音人，添加后即显示该发音人参数值，若试用未添加的发音人会报错11200
public static final String VCN = "aisjiuxu";
// json
public static final Gson gson = new Gson();
public static boolean wsCloseFlag = false;
public void setWsCloseFlag(boolean wsCloseFlag) {
WebTtsWs.wsCloseFlag = wsCloseFlag;
}
public String createVoice(String text) throws Exception {
ossUtils.init();
String wsUrl = getAuthUrl(hostUrl, apiKey, apiSecret).replace("https://", "wss://");
String basePath = System.getProperty("user.dir") + "\\voiceSource\\" + System.currentTimeMillis() + ".mp3";
File file = new File(basePath);
OutputStream outputStream = new FileOutputStream(file);
websocketWork(wsUrl, outputStream,text);
Thread.sleep(2500);
if (file.exists() && file.isFile()) {
long fileSizeInBytes = file.length();
System.out.println("文件的字节大小为: " + fileSizeInBytes + " 字节");
} else {
System.out.println("文件不存在或不是一个常规文件。");
}
String fileString = ossUtils.uploadFile(file, "audio/mpeg");
ossUtils.shutdown();
return fileString;
}
// Websocket方法，为流式文件生成服务
public static void websocketWork(String wsUrl, OutputStream outputStream,String text) {
try {
URI uri = new URI(wsUrl);
WebSocketClient webSocketClient = new WebSocketClient(uri) {
@Override
public void onOpen(ServerHandshake serverHandshake) {
System.out.println("ws建立连接成功...");
}
@Override
public void onMessage(String text) {
// System.out.println(text);
JsonParse myJsonParse = gson.fromJson(text, JsonParse.class);
System.out.println("---------------" + myJsonParse + "数据的值是这个");
if (myJsonParse.code != 0) {
System.out.println("发生错误，错误码为：" + myJsonParse.code);
System.out.println("本次请求的sid为：" + myJsonParse.sid);
}
if (myJsonParse.data != null) {
try {
byte[] textBase64Decode = Base64.getDecoder().decode(myJsonParse.data.audio);
outputStream.write(textBase64Decode);
outputStream.flush();
} catch (Exception e) {
e.printStackTrace();
}
if (myJsonParse.data.status == 2) {
try {
outputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("本次请求的sid==>" + myJsonParse.sid);
// 可以关闭连接，释放资源
// wsCloseFlag = true;
}
}
}
@Override
public void onClose(int i, String s, boolean b) {
System.out.println("ws链接已关闭，本次请求完成...");
}
@Override
public void onError(Exception e) {
System.out.println("发生错误 " + e.getMessage());
}
};
// 建立连接
webSocketClient.connect();
while (!webSocketClient.getReadyState().equals(WebSocket.READYSTATE.OPEN)) {
//System.out.println("正在连接...");
Thread.sleep(100);
}
MyThread webSocketThread = new MyThread(webSocketClient,text);
webSocketThread.start();
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
// 线程来发送音频与参数
static class MyThread extends Thread {
WebSocketClient webSocketClient;
String text;
public MyThread(WebSocketClient webSocketClient,String text) {
this.webSocketClient = webSocketClient;
this.text = text;
}
public void run() {
String requestJson;//请求参数json串
try {
requestJson = "{\n" +
" \"common\": {\n" +
" \"app_id\": \"" + appid + "\"\n" +
" },\n" +
" \"business\": {\n" +
" \"aue\": \"lame\",\n" +
" \"sfl\": 1,\n" +
" \"tte\": \"" + TTE + "\",\n" +
" \"ent\": \"intp65\",\n" +
" \"vcn\": \"" + VCN + "\",\n" +
" \"pitch\": 50,\n" +
" \"volume\": 100,\n" +
" \"speed\": 50\n" +
" },\n" +
" \"data\": {\n" +
" \"status\": 2,\n" +
" \"text\": \"" + Base64.getEncoder().encodeToString(text.getBytes(StandardCharsets.UTF_8)) + "\"\n" +
//" \"text\": \"" + Base64.getEncoder().encodeToString(TEXT.getBytes("UTF-16LE")) + "\"\n" +
" }\n" +
"}";
webSocketClient.send(requestJson);
// 等待服务端返回完毕后关闭
while (!wsCloseFlag) {
Thread.sleep(200);
}
webSocketClient.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
// 鉴权方法
public static String getAuthUrl(String hostUrl, String apiKey, String apiSecret) throws Exception {
URL url = new URL(hostUrl);
// 时间
SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US);
format.setTimeZone(TimeZone.getTimeZone("GMT"));
String date = format.format(new Date());
// 拼接
String preStr = "host: " + url.getHost() + "\n" +
"date: " + date + "\n" +
"GET " + url.getPath() + " HTTP/1.1";
//System.out.println(preStr);
// SHA256加密
Mac mac = Mac.getInstance("hmacsha256");
SecretKeySpec spec = new SecretKeySpec(apiSecret.getBytes(StandardCharsets.UTF_8), "hmacsha256");
mac.init(spec);
byte[] hexDigits = mac.doFinal(preStr.getBytes(StandardCharsets.UTF_8));
// Base64加密
String sha = Base64.getEncoder().encodeToString(hexDigits);
// 拼接
String authorization = String.format("api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"", apiKey, "hmac-sha256", "host date request-line", sha);
// 拼接地址
HttpUrl httpUrl = Objects.requireNonNull(HttpUrl.parse("https://" + url.getHost() + url.getPath())).newBuilder().//
addQueryParameter("authorization", Base64.getEncoder().encodeToString(authorization.getBytes(StandardCharsets.UTF_8))).//
addQueryParameter("date", date).//
addQueryParameter("host", url.getHost()).//
build();
return httpUrl.toString();
}
//返回的json结果拆解
class JsonParse {
int code;
String sid;
Data data;
}
class Data {
int status;
String audio;
}
}语音识别服务 IAT：
调用voiceMsgSendToAI方法，传入语音文件的MutiPartFile文件形式数据，就会返回识别结果。
iat工具类：
public class WebIATWS extends WebSocketListener {
private static final String hostUrl = "https://iat-api.xfyun.cn/v2/iat"; //中英文，http url 不支持解析 ws/wss schema
// private static final String hostUrl = "https://iat-niche-api.xfyun.cn/v2/iat";//小语种
private static final String appid = "。。。。。。。。。"; //在控制台-我的应用获取
private static final String apiSecret = "。。。。。。。。"; //在控制台-我的应用-语音听写（流式版）获取
private static final String apiKey = "。。。。。。。。。"; //在控制台-我的应用-语音听写（流式版）获取
private String filePath;
public static final int StatusFirstFrame = 0;
public static final int StatusContinueFrame = 1;
public static final int StatusLastFrame = 2;
public static final Gson json = new Gson();
Decoder decoder = new Decoder();
// 开始时间
private static Date dateBegin = new Date();
// 结束时间
private static Date dateEnd = new Date();
private static final SimpleDateFormat sdf = new SimpleDateFormat("yyy-MM-dd HH:mm:ss.SSS");
private static String resultContent = "无";
private MultipartFile multipartFile;
public WebIATWS(MultipartFile multipartFile) {
this.multipartFile = multipartFile;
}
//将录音文件发给ai
public String voiceMsgSendToAI(MultipartFile multipartFile) throws Exception {
// 构建鉴权url
String authUrl = getAuthUrl(hostUrl, apiKey, apiSecret);
OkHttpClient client = new OkHttpClient.Builder().build();
//将url中的 schema http://和https://分别替换为ws:// 和 wss://
String url = authUrl.toString().replace("http://", "ws://").replace("https://", "wss://");
//System.out.println(url);
Request request = new Request.Builder().url(url).build();
// System.out.println(client.newCall(request).execute());
//System.out.println("url===>" + url);
System.out.println(filePath);
String pathOption = StringEscapeUtils.escapeJava(filePath);
System.out.println(pathOption);
WebSocket webSocket = client.newWebSocket(request, new WebIATWS(multipartFile));
Thread.sleep(1500);
System.out.println("要发送的结果是："+resultContent);
return resultContent;
}
@Override
public void onOpen(WebSocket webSocket, Response response) {
super.onOpen(webSocket, response);
new Thread(()->{
//连接成功，开始发送数据
int frameSize = 1280; //每一帧音频的大小,建议每 40ms 发送 122B
int intervel = 40;
int status = 0; // 音频的状态
String basePath = System.getProperty("user.dir") + "\\voiceSource";
//原始文件名
String originalFilename = multipartFile.getOriginalFilename();
//获得文件名末尾的文件格式，如：.jpg
String suffix = originalFilename.substring(originalFilename.lastIndexOf("."));
//生成一个带指定文件格式的不重复的字符串做文件命名
String fileName = UUID.randomUUID() + suffix;
//根据需求判断是否要新建一个目录
File dir = new File(basePath);
if (!dir.exists()){
//如果不存在，则创建此目录
dir.mkdirs();
}
try {
//将临时文件转存到指定位置
multipartFile.transferTo(new File(basePath + "\\" + fileName));
} catch (IOException e) {
e.printStackTrace();
}
System.out.println(fileName);
String filePath = basePath + "\\" + fileName;
try (FileInputStream fs = new FileInputStream(filePath)) {
byte[] buffer = new byte[frameSize];
// 发送音频
end:
while (true) {
int len = fs.read(buffer);
if (len == -1) {
status = StatusLastFrame; //文件读完，改变status 为 2
}
switch (status) {
case StatusFirstFrame: // 第一帧音频status = 0
JsonObject frame = new JsonObject();
JsonObject business = new JsonObject(); //第一帧必须发送
JsonObject common = new JsonObject(); //第一帧必须发送
JsonObject data = new JsonObject(); //每一帧都要发送
// 填充common
common.addProperty("app_id", appid);
//填充business
business.addProperty("language", "zh_cn");
//business.addProperty("language", "en_us");//英文
//business.addProperty("language", "ja_jp");//日语，在控制台可添加试用或购买
//business.addProperty("language", "ko_kr");//韩语，在控制台可添加试用或购买
//business.addProperty("language", "ru-ru");//俄语，在控制台可添加试用或购买
business.addProperty("domain", "iat");
// business.addProperty("accent", "mandarin");//中文方言请在控制台添加试用，添加后即展示相应参数值
//business.addProperty("nunum", 0);
//business.addProperty("ptt", 0);//标点符号
// business.addProperty("rlang", "zh-hk"); // zh-cn :简体中文（默认值）zh-hk :繁体香港(若未授权不生效，在控制台可免费开通)
//business.addProperty("vinfo", 1);
business.addProperty("dwa", "wpgs");//动态修正(若未授权不生效，在控制台可免费开通)
business.addProperty("nbest", 5);// 句子多候选(若未授权不生效，在控制台可免费开通)
business.addProperty("wbest", 3);// 词级多候选(若未授权不生效，在控制台可免费开通)
//填充data
data.addProperty("status", StatusFirstFrame);
data.addProperty("format", "audio/L16;rate=16000");
data.addProperty("encoding", "lame");
data.addProperty("audio", Base64.getEncoder().encodeToString(Arrays.copyOf(buffer, len)));
//填充frame
frame.add("common", common);
frame.add("business", business);
frame.add("data", data);
webSocket.send(frame.toString());
status = StatusContinueFrame; // 发送完第一帧改变status 为 1
break;
case StatusContinueFrame: //中间帧status = 1
JsonObject frame1 = new JsonObject();
JsonObject data1 = new JsonObject();
data1.addProperty("status", StatusContinueFrame);
data1.addProperty("format", "audio/L16;rate=16000");
data1.addProperty("encoding", "lame");
data1.addProperty("audio", Base64.getEncoder().encodeToString(Arrays.copyOf(buffer, len)));
frame1.add("data", data1);
webSocket.send(frame1.toString());
// System.out.println("send continue");
break;
case StatusLastFrame: // 最后一帧音频status = 2 ，标志音频发送结束
JsonObject frame2 = new JsonObject();
JsonObject data2 = new JsonObject();
data2.addProperty("status", StatusLastFrame);
data2.addProperty("audio", "");
data2.addProperty("format", "audio/L16;rate=16000");
data2.addProperty("encoding", "lame");
frame2.add("data", data2);
webSocket.send(frame2.toString());
System.out.println("sendlast");
break end;
}
Thread.sleep(intervel); //模拟音频采样延时
}
System.out.println("all data is send");
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
}
}).start();
}
@Override
public void onMessage(WebSocket webSocket, String text) {
super.onMessage(webSocket, text);
//System.out.println(text);
ResponseData resp = json.fromJson(text, ResponseData.class);
if (resp != null) {
if (resp.getCode() != 0) {
System.out.println( "code=>" + resp.getCode() + " error=>" + resp.getMessage() + " sid=" + resp.getSid());
System.out.println( "错误码查询链接：https://www.xfyun.cn/document/error-code");
return;
}
if (resp.getData() != null) {
if (resp.getData().getResult() != null) {
Text te = resp.getData().getResult().getText();
//System.out.println(te.toString());
try {
decoder.decode(te);
System.out.println("中间识别结果 ==》" + decoder.toString());
} catch (Exception e) {
e.printStackTrace();
}
}
if (resp.getData().getStatus() == 2) {
// todo resp.data.status ==2 说明数据全部返回完毕，可以关闭连接，释放资源
System.out.println("session end ");
dateEnd = new Date();
System.out.println(sdf.format(dateBegin) + "开始");
System.out.println(sdf.format(dateEnd) + "结束");
System.out.println("耗时:" + (dateEnd.getTime() - dateBegin.getTime()) + "ms");
System.out.println("最终识别结果 ==》" + decoder.toString());
resultContent = decoder.toString();
System.out.println(resultContent+"########");
System.out.println("本次识别sid ==》" + resp.getSid());
decoder.discard();
webSocket.close(1000, "");
} else {
// todo 根据返回的数据处理
resultContent = decoder.toString();
}
}
}
}
@Override
public void onFailure(WebSocket webSocket, Throwable t, Response response) {
super.onFailure(webSocket, t, response);
try {
if (null != response) {
int code = response.code();
System.out.println("onFailure code:" + code);
System.out.println("onFailure body:" + response.body().string());
if (101 != code) {
System.out.println("connection failed");
System.exit(0);
}
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static String getAuthUrl(String hostUrl, String apiKey, String apiSecret) throws Exception {
URL url = new URL(hostUrl);
SimpleDateFormat format = new SimpleDateFormat("EEE, dd MMM yyyy HH:mm:ss z", Locale.US);
format.setTimeZone(TimeZone.getTimeZone("GMT"));
String date = format.format(new Date());
StringBuilder builder = new StringBuilder("host: ").append(url.getHost()).append("\n").//
append("date: ").append(date).append("\n").//
append("GET ").append(url.getPath()).append(" HTTP/1.1");
//System.out.println(builder);
Charset charset = Charset.forName("UTF-8");
Mac mac = Mac.getInstance("hmacsha256");
SecretKeySpec spec = new SecretKeySpec(apiSecret.getBytes(charset), "hmacsha256");
mac.init(spec);
byte[] hexDigits = mac.doFinal(builder.toString().getBytes(charset));
String sha = Base64.getEncoder().encodeToString(hexDigits);
//System.out.println(sha);
String authorization = String.format("api_key=\"%s\", algorithm=\"%s\", headers=\"%s\", signature=\"%s\"", apiKey, "hmac-sha256", "host date request-line", sha);
//System.out.println(authorization);
HttpUrl httpUrl = HttpUrl.parse("https://" + url.getHost() + url.getPath()).newBuilder().//
addQueryParameter("authorization", Base64.getEncoder().encodeToString(authorization.getBytes(charset))).//
addQueryParameter("date", date).//
addQueryParameter("host", url.getHost()).//
build();
return httpUrl.toString();
}
public static class ResponseData {
private int code;
private String message;
private String sid;
private Data data;
public int getCode() {
return code;
}
public String getMessage() {
return this.message;
}
public String getSid() {
return sid;
}
public Data getData() {
return data;
}
}
public static class Data {
private int status;
private Result result;
public int getStatus() {
return status;
}
public Result getResult() {
return result;
}
}
public static class Result {
int bg;
int ed;
String pgs;
int[] rg;
int sn;
Ws[] ws;
boolean ls;
JsonObject vad;
public Text getText() {
Text text = new Text();
StringBuilder sb = new StringBuilder();
for (Ws ws : this.ws) {
sb.append(ws.cw[0].w);
}
text.sn = this.sn;
text.text = sb.toString();
text.sn = this.sn;
text.rg = this.rg;
text.pgs = this.pgs;
text.bg = this.bg;
text.ed = this.ed;
text.ls = this.ls;
text.vad = this.vad==null ? null : this.vad;
return text;
}
}
public static class Ws {
Cw[] cw;
int bg;
int ed;
}
public static class Cw {
int sc;
String w;
}
public static class Text {
int sn;
int bg;
int ed;
String text;
String pgs;
int[] rg;
boolean deleted;
boolean ls;
JsonObject vad;
@Override
public String toString() {
return "Text{" +
"bg=" + bg +
", ed=" + ed +
", ls=" + ls +
", sn=" + sn +
", text='" + text + '\'' +
", pgs=" + pgs +
", rg=" + Arrays.toString(rg) +
", deleted=" + deleted +
", vad=" + (vad==null ? "null" : vad.getAsJsonArray("ws").toString()) +
'}';
}
}
//解析返回数据，仅供参考
public static class Decoder {
private Text[] texts;
private int defc = 10;
public Decoder() {
this.texts = new Text[this.defc];
}
public synchronized void decode(Text text) {
if (text.sn >= this.defc) {
this.resize();
}
if ("rpl".equals(text.pgs)) {
for (int i = text.rg[0]; i <= text.rg[1]; i++) {
this.texts[i].deleted = true;
}
}
this.texts[text.sn] = text;
}
public String toString() {
StringBuilder sb = new StringBuilder();
for (Text t : this.texts) {
if (t != null && !t.deleted) {
sb.append(t.text);
}
}
return sb.toString();
}
public void resize() {
int oc = this.defc;
this.defc <<= 1;
Text[] old = this.texts;
this.texts = new Text[this.defc];
for (int i = 0; i < oc; i++) {
this.texts[i] = old[i];
}
}
public void discard(){
for(int i=0;i<this.texts.length;i++){
this.texts[i]= null;
}
}
}
}使用案例：
@PostMapping("/audioAnalysis")
public ResultData<String> audioAnalysis(@RequestParam("file") MultipartFile file) throws Exception {
WebIATWS webIATWS = new WebIATWS(file);
String resultString = webIATWS.voiceMsgSendToAI(file);
return ResultData.success(resultString);
}tuputech表情识别服务：
要多放两个util类：
public class FileUtil {
/**
* 读取文件内容为二进制数组
*
* @param filePath
* @return
* @throws IOException
*/
public static byte[] read(String filePath) throws IOException {
InputStream in = new FileInputStream(filePath);
byte[] data = inputStream2ByteArray(in);
in.close();
return data;
}
/**
* 流转二进制数组
*
* @param in
* @return
* @throws IOException
*/
private static byte[] inputStream2ByteArray(InputStream in) throws IOException {
ByteArrayOutputStream out = new ByteArrayOutputStream();
byte[] buffer = new byte[1024 * 4];
int n = 0;
while ((n = in.read(buffer)) != -1) {
out.write(buffer, 0, n);
}
return out.toByteArray();
}
/**
* 保存文件
*
* @param filePath
* @param fileName
* @param content
*/
public static void save(String filePath, String fileName, byte[] content) {
try {
File filedir = new File(filePath);
if (!filedir.exists()) {
filedir.mkdirs();
}
File file = new File(filedir, fileName);
OutputStream os = new FileOutputStream(file);
os.write(content, 0, content.length);
os.flush();
os.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}public class HttpUtil {
/**
* 发送post请求
*
* @param url
* @param header
* @param body
* @return
*/
public static String doPost1(String url, Map<String, String> header, byte[] body) {
String result = "";
BufferedReader in = null;
try {
// 设置 url
URL realUrl = new URL(url);
URLConnection connection = realUrl.openConnection();
HttpURLConnection httpURLConnection = (HttpURLConnection) connection;
// 设置 header
for (String key : header.keySet()) {
httpURLConnection.setRequestProperty(key, header.get(key));
}
// 设置请求 body
httpURLConnection.setDoOutput(true);
httpURLConnection.setDoInput(true);
httpURLConnection.setRequestProperty("Content-Type", "binary/octet-stream");
OutputStream out = httpURLConnection.getOutputStream();
out.write(body);
out.flush();
out.close();
if (HttpURLConnection.HTTP_OK != httpURLConnection.getResponseCode()) {
System.out.println("Http 请求失败，状态码：" + httpURLConnection.getResponseCode());
return null;
}
// 获取响应body
in = new BufferedReader(new InputStreamReader(httpURLConnection.getInputStream()));
String line;
while ((line = in.readLine()) != null) {
result += line;
}
} catch (Exception e) {
return null;
}
return result;
}
}表情识别工具类：
调用方法getFaceExpression传入文件名（任意），传入表情图片文件的MutiPartFile文件数据，即可返回一个表情识别数据。
/**
*人脸特征分析表情WebAPI接口调用示例接口文档(必看)：https://doc.xfyun.cn/rest_api/%E4%BA%BA%E8%84%B8%E7%89%B9%E5%BE%81%E5%88%86%E6%9E%90-%E8%A1%A8%E6%83%85.html
*图片属性：png、jpg、jpeg、bmp、tif图片大小不超过800k
*(Very Important)创建完webapi应用添加服务之后一定要设置ip白名单，找到控制台--我的应用--设置ip白名单，如何设置参考：http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=41891
*错误码链接：https://www.xfyun.cn/document/error-code (code返回错误码时必看)
*@author iflytek
*/
public class face {
// webapi 接口地址
private static final String URL = "http://tupapi.xfyun.cn/v1/expression";
// 应用ID(必须为webapi类型应用,并人脸特征分析服务,参考帖子如何创建一个webapi应用：http://bbs.xfyun.cn/forum.php?mod=viewthread&tid=36481)
private static final String APPID = "。。。。。。。";
// 接口密钥(webapi类型应用开通人脸特征分析服务后，控制台--我的应用---人脸特征分析---服务的apikey
private static final String API_KEY = "。。。。。。。";
// 图片数据可以通过两种方式上传，第一种在请求头设置image_url参数，第二种将图片二进制数据写入请求体中。若同时设置，以第一种为准。
// 此demo使用第二种方式进行上传图片地址，如果想使用第一种方式，请求体为空即可。
// 图片名称
private static final String IMAGE_NAME = "1.jpg";
// 图片url
//private static final String IMAGE_URL = " ";
// 图片地址
private static final String PATH = "image/1.jpg";
/**
* WebAPI 调用示例程序
*
* @param args
* @throws IOException
*/
public String getFaceExpression(String imageName, MultipartFile file) throws IOException {
Map<String, String> header = buildHttpHeader(imageName,filePath);
byte[] imageByteArray = file.getBytes();
String result = HttpUtil.doPost1(URL,header, imageByteArray);
System.out.println("接口调用结果：" + result);
return result;
}
/**
* 组装http请求头
*/
private static Map<String, String> buildHttpHeader(String imageName) throws UnsupportedEncodingException {
String curTime = System.currentTimeMillis() / 1000L + "";
String param = "{\"image_name\":\"" + imageName + "\"}";
String paramBase64 = new String(Base64.encodeBase64(param.getBytes("UTF-8")));
String checkSum = DigestUtils.md5Hex(API_KEY + curTime + paramBase64);
Map<String, String> header = new HashMap<String, String>();
header.put("Content-Type", "application/x-www-form-urlencoded; charset=utf-8");
header.put("X-Param", paramBase64);
header.put("X-CurTime", curTime);
header.put("X-CheckSum", checkSum);
header.put("X-Appid", APPID);
return header;
}
}结果中信息的意思如下
参数字段说明：
rate ：介于0-1间的浮点数，表示该图像被识别为某个分类的概率值，概率越高、机器越肯定
rates：各个label分别对应的概率值的数组，顺序如label的大小从小到大的顺序
label：大于等于0时，表明图片属于哪个分类或结果；等于-1时，代表该图片文件有错误，或者格式不支持（gif图不支持）
name：图片的url地址或名称
review：本次识别结果是否存在偏差，返回true时存在偏差，可信度较低，返回false时可信度较高，具体可参考rate参数值
fileList：每张图片的识别结果
reviewCount：需要复审的图片数量
statistic：各个分类的图片数量
label的值及其对应的表情
0：其他(非人脸表情图片)
1：其他表情
2：喜悦
3：愤怒
4：悲伤
5：惊恐
6：厌恶
7：中性使用案例：
@PostMapping("/expressionAnalysis")
public String expressionAnalysis(@RequestParam("file") MultipartFile file) throws IOException {
ossUtils.init();
String url = ossUtils.uploadFile(file);
ossUtils.shutdown();
System.out.println(url);
String[] split = url.split("/");
String fileName = split[split.length - 1];
face face = new face();
String faceExpression = face.getFaceExpression(fileName, url,file);
return faceExpression;
}
@PostMapping("/expressionAnalysisResult")
public ResultData<String> expressionAnalysisResult(@RequestBody FileDetail[] fileDetail) throws IOException {
String expressionData = JSON.toJSONString(fileDetail);
Long userId = LearningThreadLocalUtil.getUser();
RoleContentPo roleContentPo = new RoleContentPo();
roleContentPo.setRole("user");
roleContentPo.setRcid(12345678L);
roleContentPo.setScid(111111111L);
roleContentPo.setUid("expressionAnalysis" + System.currentTimeMillis() + "-" + userId);
roleContentPo.setContent(expressionData);
roleContentPo.setSendTime(new Date());
String background = """
你是一个面试建议小助手，你要根据用户的面试数据，温柔地鼓励用户，温柔地给用户一些建议。不要太死板，要温和。
用户会给你一个表情数据的json格式的字符串，里面包含了面试过程中抓拍到的表情分析的结果集，你需要根据这个结果集，给出合理的分析和建议。
这些是表情数据的json格式的说明：
参数字段说明：
rate ：介于0-1间的浮点数，表示该图像被识别为某个分类的概率值，概率越高、机器越肯定
rates：各个label分别对应的概率值的数组，顺序如label的大小从小到大的顺序
label：大于等于0时，表明图片属于哪个分类或结果；等于-1时，代表该图片文件有错误，或者格式不支持（gif图不支持）
name：图片的url地址或名称
review：本次识别结果是否存在偏差，返回true时存在偏差，可信度较低，返回false时可信度较高，具体可参考rate参数值
label的值及其对应的表情
0：其他(非人脸表情图片)
1：其他表情
2：喜悦
3：愤怒
4：悲伤
5：惊恐
6：厌恶
7：中性
注意：最终的结果中，不用包含整体分析，我只要得到给用户的本次面试的建议即可。记住你是一个平易近人的面试小助手。
""";
ResultData resultData = pushService.pushMessageToXFServer(roleContentPo, background);
RoleContentPo data = (RoleContentPo) resultData.getData();
String content = data.getContent();
return ResultData.success(content);
}