How should I write the .i file to wrap callbacks in Java or C#

You can do this if you have a chance to pass some data around with the callback, but you’ll need to write some JNI glue. I put together a complete example of how you might map C style callbacks onto a Java interface.

The first thing you need to do is decide on an interface that’s appropriate on the Java side. I assumed in C we had callbacks like:

typedef void (*callback_t)(int arg, void *userdata);

I decided to represent that in Java as:

public interface Callback {
  public void handle(int value);
}

(The loss of the void *userdata on the Java side isn’t a real problem since we can store state in the Object that implements Callback trivially).

I then wrote the following header file (it shouldn’t really be just a header, but it keeps things simple) to exercise the wrapping:

typedef void (*callback_t)(int arg, void *data);

static void *data = NULL;
static callback_t active = NULL;

static void set(callback_t cb, void *userdata) {
  active = cb;
  data = userdata;
}

static void dispatch(int val) {
  active(val, data);
}

I was able to successfully wrap this C with the following interface:

%module test

%{
#include <assert.h>
#include "test.h"

// 1:
struct callback_data {
  JNIEnv *env;
  jobject obj;
};

// 2:
void java_callback(int arg, void *ptr) {
  struct callback_data *data = ptr;
  const jclass callbackInterfaceClass = (*data->env)->FindClass(data->env, "Callback");
  assert(callbackInterfaceClass);
  const jmethodID meth = (*data->env)->GetMethodID(data->env, callbackInterfaceClass, "handle", "(I)V");
  assert(meth);
  (*data->env)->CallVoidMethod(data->env, data->obj, meth, (jint)arg);
}
%}

// 3:
%typemap(jstype) callback_t cb "Callback";
%typemap(jtype) callback_t cb "Callback";
%typemap(jni) callback_t cb "jobject";
%typemap(javain) callback_t cb "$javainput";
// 4:
%typemap(in,numinputs=1) (callback_t cb, void *userdata) {
  struct callback_data *data = malloc(sizeof *data);
  data->env = jenv;
  data->obj = JCALL1(NewGlobalRef, jenv, $input);
  JCALL1(DeleteLocalRef, jenv, $input);
  $1 = java_callback;
  $2 = data;
}

%include "test.h"

The interface has quite a few parts to it:

1. A struct to store the information needed to make a call to the Java interface.
2. An implementation of the callback_t. It accepts as user data the struct we just defined and then dispatches a call to the Java interface using some standard JNI.
3. Some typemaps that cause the Callback objects to be passed straight to the C implementation as a real jobject.
4. A typemap that hides the void* on the Java side and sets up a callback data and fills in the corresponding arguments for the real function to use the function we just wrote for dispatching calls back to Java. It takes a global reference to the Java object to prevent it from being garbage collected subsequently.

I wrote a little Java class to test it with:

public class run implements Callback {
  public void handle(int val) {
    System.out.println("Java callback - " + val);
  }

  public static void main(String argv[]) {
    run r = new run();

    System.loadLibrary("test");
    test.set(r);
    test.dispatch(666);    
  }
}

which worked as you’d hope.

Some points to note:

1. If you call set multiple times it will leak the global reference. You either need to supply a way for the callback to be unset, prevent setting multiple times, or use weak references instead.
2. If you have multiple threads around you will need to be smarter with the JNIEnv than I’ve been here.
3. If you wanted to mix Java and C implementations of callbacks then you’ll need to expand on this solution quite a bit. You can expose C functions to be used as callbacks with %constant but these typemaps will prevent your wrapped functions from accepting such inputs. Probably you would want to supply overloads to work around that.

There’s some more good advice in this question.

I believe that a solution for C# would be somewhat similar, with different typemap names and a differing implementation of the callback function you write in C.