If you replace ‘ by unicode encoded ‘ (which is \u0027), then it works:
curl http://dumbdomain.com/solr/collection2/update/json -H 'Content-type:application/json' -d ' { "add": { "doc": { "uid": "79729", "text" : "I\u0027ve got your number"} } }'
Strange, but worth to know!