Flink1.20.1集成Paimon遇到的问题-EW帮帮网

flinkcdc mysql 到paimon

1：Caused by: java.lang.ClassNotFoundException: org.apache.kafka.connect.data.Schema

可以参考这个文章明确指出了flink-connector-mysql-cdc-3.4.0.jar存在这个包，但是flink-sql-connector-mysql-cdc-3.4.0.jar中没有这个包可以在maven仓库中看到
在这里插入图片描述

https://developer.aliyun.com/ask/574255?spm=a2c6h.12873639.article-detail.8.6a82fe85SbDdj5

缺少 kafka的 connect-api-3.3.2.jar

Caused by: java.lang.NoClassDefFoundError: org/apache/kafka/connect/data/Schema
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.getDeclaredMethod(Class.java:2128)
at java.io.ObjectStreamClass.getPrivateMethod(ObjectStreamClass.java:1575)
at java.io.ObjectStreamClass.access$1700(ObjectStreamClass.java:79)
at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:508)
at java.io.ObjectStreamClasscreateJobVertex $24 (St re amin g J o b G r a p h G e n er a t or . ja v a : 1021) a t ja v a . u t i l . co n c u rre n t . C o m pl e t ab l e F u t u re$ AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor $W or k er . r u n (T h re a d P oo lE x ec u t or . ja v a : 624) ...1 m ore C a u se d b y : ja v a . l an g . Cl a ss N o tF o u n d E x ce pt i o n : or g . a p a c h e . ka f ka . co nn ec t . d a t a . S c h e maa t ja v a . n e t . U R L Cl a ss L o a d er . f in d Cl a ss (U R L Cl a ss L o a d er . ja v a : 381) a t ja v a . l an g . Cl a ss L o a d er . l o a d Cl a ss (Cl a ss L o a d er . ja v a : 424) a t s u n . mi sc . L a u n c h er$ AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
… 45 more

2:Caused by: java.io.InvalidClassException: org.apache.flink.cdc.connectors.shaded.com.fasterxml.jackson.databind.cfg.MapperConfig; incompatible types for field _mapperFeatures

不能同时存在
flink-connector-mysql-cdc-3.4.0.jar
flink-sql-connector-mysql-cdc-3.4.0.jar

3:cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.userFunction of type org.apache.flink.api.common.functions.Function in instance of org.apache.flink.streaming.api.operators.StreamFlatMap

dinky standalone 提mysqlCDC整库同步到paimon
将dinky-app的依赖丢进flink lib中就可以了

025-07-10 10:50:36,319 WARN  org.apache.flink.runtime.taskmanager.Task                    [] - Source: MySQL CDC Source -> PartitionByPrimaryKey -> Shunt -> FlatMapRow -> *anonymous_datastream_source$6*[22] -> Calc[23] -> ConstraintEnforcer[24] -> Map (1/2)#24 (2548d4605e02b73a83d284ff33f2e106_cbc357ccb763df2852fee8c4fc7d55f2_0_24) switched from INITIALIZING to FAILED with failure cause:
org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot instantiate user function.
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:416) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperator(OperatorChain.java:869) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperatorChain(OperatorChain.java:836) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:732) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperatorChain(OperatorChain.java:825) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:732) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOperatorChain(OperatorChain.java:825) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:732) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>(OperatorChain.java:202) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.<init>(RegularOperatorChain.java:60) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:789) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:970) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:939) [flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:763) [flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) [flink-dist-1.20.1.jar:1.20.1]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: java.lang.ClassCastException: cannot assign instance of java.lang.invoke.SerializedLambda to field org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.userFunction of type org.apache.flink.api.common.functions.Function in instance of org.apache.flink.streaming.api.operators.StreamFlatMap
	at java.io.ObjectStreamClass$FieldReflector.setObjFieldValues(ObjectStreamClass.java:2233) ~[?:1.8.0_171]
	at java.io.ObjectStreamClass.setObjFieldValues(ObjectStreamClass.java:1405) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2291) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) ~[?:1.8.0_171]
	at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) ~[?:1.8.0_171]
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:488) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:472) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:467) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:422) ~[flink-dist-1.20.1.jar:1.20.1]
	at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperatorFactory(StreamConfig.java:400) ~[flink-dist-1.20.1.jar:1.20.1]
	... 16 more

Flink1.20.1集成Paimon遇到的问题

网站公告

今日签到

热门文章

最新发布