What is a simple, effective way to debug custom Kafka connectors?

I will try to reply to your question in a broad way. A simple way to do Connector development could be as follows:

• Structure and build your connector source code by looking at one of the many Kafka Connectors available publicly (you’ll find an extensive list available here: https://www.confluent.io/product/connectors/ )
• Download the latest Confluent Open Source edition (>= 3.3.0) from https://www.confluent.io/download/

• Make your connector package available to Kafka Connect in one of the following ways:

  1. Store all your connector jar files (connector jar plus dependency jars excluding Connect API jars) to a location in your filesystem and enable plugin isolation by adding this location to the
     plugin.path property in the Connect worker properties. For instance, if your connector jars are stored in /opt/connectors/my-first-connector, you will set plugin.path=/opt/connectors in your worker’s properties (see below).
  2. Store all your connector jar files in a folder under ${CONFLUENT_HOME}/share/java. For example: ${CONFLUENT_HOME}/share/java/kafka-connect-my-first-connector. (Needs to start with kafka-connect- prefix to be picked up by the startup scripts). $CONFLUENT_HOME is where you’ve installed Confluent Platform.

• Optionally, increase your logging by changing the log level for Connect in ${CONFLUENT_HOME}/etc/kafka/connect-log4j.properties to DEBUG or even TRACE.

• Use Confluent CLI to start all the services, including Kafka Connect. Details here: http://docs.confluent.io/current/connect/quickstart.html

  Briefly: confluent start

Note: The Connect worker’s properties file currently loaded by the CLI is ${CONFLUENT_HOME}/etc/schema-registry/connect-avro-distributed.properties. That’s the file you should edit if you choose to enable classloading isolation but also if you need to change your Connect worker’s properties.

• Once you have Connect worker running, start your connector by running:

  confluent load <connector_name> -d <connector_config.properties>

  or

  confluent load <connector_name> -d <connector_config.json>

  The connector configuration can be either in java properties or JSON format.

• Run
  confluent log connect to open the Connect worker’s log file, or navigate directly to where your logs and data are stored by running

  cd "$( confluent current )"

Note: change where your logs and data are stored during a session of the Confluent CLI by setting the environment variable CONFLUENT_CURRENT appropriately. E.g. given that /opt/confluent exists and is where you want to store your data, run:

export CONFLUENT_CURRENT=/opt/confluent
confluent current

• Finally, to interactively debug your connector a possible way is to apply the following before starting Connect with Confluent CLI :

  confluent stop connect
export CONNECT_DEBUG=y; export DEBUG_SUSPEND_FLAG=y;
confluent start connect

  and then connect with your debugger (for instance remotely to the Connect worker (default port: 5005). To stop running connect in debug mode, just run: unset CONNECT_DEBUG; unset DEBUG_SUSPEND_FLAG; when you are done.

I hope the above will make your connector development easier and … more fun!