Webhook data to Kafka Connect

I'm using Kafka connect to pull data from different places: mysql, mongodb, etc. And send to elastic search.

I would like to pull data where the origin is a webhook. So, can I configure some Kafka URL to send post http requests to it? What is a good practice to send to my Kafka http post requests?

Solution 1:

Confluent maintains a source-available Kafka REST Proxy (be sure to read the Confluent Community License of the project). This would allow you to "send POST requests" to something that forwards this data to Kafka. Outside of this, write a simple HTTP endpoint on your own that does the same.

Personally, I have used Nifi's ListenHTTP handler to accept webhooks, then parse, route, filter, etc into a ProduceKafka request

Otherwise "pulling data" isn't a web hook, and there are a handful of "Kafka Connect http" source projects on Github.