After a lot research and testing, I found that “Manager” does this job at a non-complex object level.
The code below shows that object inst is shared between processes, which means property var of inst is changed outside when child process changes it.
from multiprocessing import Process, Manager
from multiprocessing.managers import BaseManager
class SimpleClass(object):
def __init__(self):
self.var = 0
def set(self, value):
self.var = value
def get(self):
return self.var
def change_obj_value(obj):
obj.set(100)
if __name__ == '__main__':
BaseManager.register('SimpleClass', SimpleClass)
manager = BaseManager()
manager.start()
inst = manager.SimpleClass()
p = Process(target=change_obj_value, args=[inst])
p.start()
p.join()
print inst # <__main__.SimpleClass object at 0x10cf82350>
print inst.get() # 100
Okay, above code is enough if you only need to share simple objects.
Why no complex? Because it may fail if your object is nested (object inside object):
from multiprocessing import Process, Manager
from multiprocessing.managers import BaseManager
class GetSetter(object):
def __init__(self):
self.var = None
def set(self, value):
self.var = value
def get(self):
return self.var
class ChildClass(GetSetter):
pass
class ParentClass(GetSetter):
def __init__(self):
self.child = ChildClass()
GetSetter.__init__(self)
def getChild(self):
return self.child
def change_obj_value(obj):
obj.set(100)
obj.getChild().set(100)
if __name__ == '__main__':
BaseManager.register('ParentClass', ParentClass)
manager = BaseManager()
manager.start()
inst2 = manager.ParentClass()
p2 = Process(target=change_obj_value, args=[inst2])
p2.start()
p2.join()
print inst2 # <__main__.ParentClass object at 0x10cf82350>
print inst2.getChild() # <__main__.ChildClass object at 0x10cf6dc50>
print inst2.get() # 100
#good!
print inst2.getChild().get() # None
#bad! you need to register child class too but there's almost no way to do it
#even if you did register child class, you may get PicklingError :)
I think the main reason of this behavior is because Manager is just a candybar built on top of low-level communication tools like pipe/queue.
So, this approach is not well recommended for multiprocessing case. It’s always better if you can use low-level tools like lock/semaphore/pipe/queue or high-level tools like Redis queue or Redis publish/subscribe for complicated use case (only my recommendation lol).