Has some Lua engine with function like:

SetUpdateCallback(func);

Try to invoke it via luabind from C++:

call_function<bool>(state, "SetUpdateCallback", &MyHandler::OnUpdate);

In runtime I get an error on the line. Looks like I should to wrap my pointer to something. Can anyone tried to pass callbacks via Luabind?

What is the exact and complete text of the error you are seeing?

Error code is 0x80004005

at _CxxThrowException(Void* , _s__ThrowInfo* )
   at luabind.detail.make_instance<class std::auto_ptr<void (__thiscall MyHandler::*)(int)> >(lua_State* L, auto_ptr<void (__thiscall MyHandler::\*)(int)>* p) in c:\luabind\detail\make_instance.hpp:line 80
   at luabind.detail.make_pointee_instance<void (__thiscall MyHandler::*)(int)>(lua_State* L, $PTMType$P8MyHandler@@AEXH@Z* x, bool_<0> __unnamed002, bool_<1> __unnamed003) in c:\luabind\detail\policy.hpp:line 183
   at luabind.detail.make_pointee_instance<void (__thiscall MyHandler::*)(int),struct boost::mpl::bool_<1> >(lua_State* L, $PTMType$P8MyHandler@@AEXH@Z* x, bool_<1> __unnamed002) in c:\luabind\detail\policy.hpp:line 195
   at luabind.detail.value_converter.apply<void (__thiscall MyHandler::*)(int)>(value_converter* , lua_State* L, $PTMType$P8MyHandler@@AEXH@Z x) in c:\luabind\detail\policy.hpp:line 281
   at luabind.detail.convert_to_lua<void (__thiscall MyHandler::*)(int)>(lua_State* L, $PTMType$Q8MyHandler@@AEXH@Z* v) in c:\luabind\detail\convert_to_lua.hpp:line 73
   at luabind.detail.push_args_from_tuple<2>.apply<void (__thiscall MyHandler::*const *)(int),struct boost::tuples::null_type>(lua_State* L, cons<void (__thiscall MyHandler::\*const \*)(int)\,boost::tuples::null_type>* x) in c:\luabind\object.hpp:line 1079
   at luabind.detail.push_args_from_tuple<1>.apply<char const (*)[18],struct boost::tuples::cons<void (__thiscall MyHandler::*const *)(int),struct boost::tuples::null_type> >(lua_State* L, cons<char const (\*)\[18\]\,boost::tuples::cons<void (__thiscall MyHandler::\*const \*)(int)\,boost::tuples::null_type> >* x) in c:\luabind\object.hpp:line 1080
   at luabind.detail.proxy_function_caller<bool,boost::tuples::tuple<char const (*)[18],void (__thiscall MyHandler::*const *)(int),boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type,boost::tuples::null_type> >.._N(proxy_function_caller<bool\,boost::tuples::tuple<char const (\*)\[18\]\,void (__thiscall MyHandler::\*const \*)(int)\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type\,boost::tuples::null_type> >* ) in c:\luabind\detail\call_function.hpp:line 121
   at MyHandler.{ctor}(MyHandler* , object* state) in c:\mylua\mylua.cpp:line 172

Looks the issue not in callbacks. The issue in Lua instances. state is an instance of Lua class. I get the same error while try to invoke any other methods (neither callbacks nor without).

Is the any workaround to work with lua instances via luabind?

Have you inspected the exception in a debugger to see where in code it originates, and what information it carries? Probably some useful hints there.

38 minutes ago, ApochPiQ said:

Have you inspected the exception in a debugger to see where in code it originates, and what information it carries? Probably some useful hints there.

Well, it a thousand of lines and not sure I've got the details in reasonable time.

How anyone used Luabind before?

I've used LuaBind before. Years ago, before it essentially died. Most of the clones or forks on github haven't seen a commit in 3 or 4 years. The Rasterbar repo hasn't seen a commit in at least that long. Plenty of time for that thing to rot, and it was never a very well-behaved library to begin with, at least in my experience. There are newer and much nicer (and far more actively developed) libraries to choose from now, such as sol2, Selene, lua-intf, Kaguya, etc... Many of them take advantage of C++11 and above, to eliminate the need for Boost dependencies. There is a fork of Luabind that eliminates the Boost dependency, and it has seen a small surge of recent commits in case you're interested. https://github.com/decimad/luabind-deboostified Other than that, I'd recommend going with a binding library that has more recent support, and an active community using it that can help in the case of weird errors.