Crash reports

Post here your questions about the C++ API for SFS2X

Moderators: Lapo, Bax, MBagnati

dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Crash reports

Postby dhuang11 » 16 Sep 2015, 12:37

Using the C++ API, we are seeing 3 types of crashes being reported on our iOS app using SFS2X 1.1.6:

Code: Select all

Thread : Crashed: Thread
0  A iOS                      0x0000000100fa3cc8 boost::asio::detail::kqueue_reactor::start_op(int, int, boost::asio::detail::kqueue_reactor::descriptor_state*&, boost::asio::detail::reactor_op*, bool, bool) (kqueue_reactor.ipp:193)
1  A iOS                      0x0000000100fa3874 boost::asio::detail::reactive_socket_service_base::start_op(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, int, boost::asio::detail::reactor_op*, bool, bool, bool) (reactive_socket_service_base.ipp:213)
2  A iOS                      0x0000000100fa363c void boost::asio::detail::reactive_socket_service_base::async_send<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, boost::asio::mutable_buffers_1 const&, int, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&) (reactive_socket_service_base.hpp:216)
3  A iOS                      0x0000000100fa347c boost::asio::async_result<boost::asio::handler_type<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::stream_socket_service<boost::asio::ip::tcp>::async_send<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::mutable_buffers_1 const&, int, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&&) (stream_socket_service.hpp:330)
4  A iOS                      0x0000000100fa28ec boost::asio::async_result<boost::asio::handler_type<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::basic_stream_socket<boost::asio::ip::tcp, boost::asio::stream_socket_service<boost::asio::ip::tcp> >::async_write_some<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::mutable_buffers_1 const&, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&&) (basic_stream_socket.hpp:731)
5  A iOS                      0x0000000100fa1020 Sfs2X::Core::Sockets::TCPClient::OnBoostAsioDataSent(boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&) (TCPClient.cpp:326)
6  A iOS                      0x0000000100fa4da0 boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>::operator()(Sfs2X::Core::Sockets::TCPClient*, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&) const (mem_fn_template.hpp:506)
7  A iOS                      0x0000000100fa4ca0 void boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()>::operator()<boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list2<boost::system::error_code const&, unsigned long const&> >(boost::_bi::type<void>, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>&, boost::_bi::list2<boost::system::error_code const&, unsigned long const&>&, int) (bind.hpp:525)
8  A iOS                      0x0000000100fa4b54 void boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >::operator()<boost::system::error_code, unsigned long>(boost::system::error_code const&, unsigned long const&) (bind_template.hpp:102)
9  A iOS                      0x0000000100fa4af0 boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long>::operator()() (bind_handler.hpp:127)
10 A iOS                      0x0000000100fa4ac4 void boost::asio::asio_handler_invoke<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long> >(boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long>&, ...) (handler_invoke_hook.hpp:69)
11 A iOS                      0x0000000100fa4a3c void boost_asio_handler_invoke_helpers::invoke<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long>, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long>&, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&) (handler_invoke_helpers.hpp:37)
12 A iOS                      0x0000000100fa4354 boost::asio::detail::reactive_socket_send_op<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) (reactive_socket_send_op.hpp:107)
13 A iOS                      0x0000000100fe7a80 boost::asio::detail::task_io_service_operation::complete(boost::asio::detail::task_io_service&, boost::system::error_code const&, unsigned long) (task_io_service_operation.hpp:38)
14 A iOS                      0x0000000100fe6ff0 boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) (task_io_service.ipp:384)
15 A iOS                      0x0000000100fe6bb0 boost::asio::detail::task_io_service::run(boost::system::error_code&) (task_io_service.ipp:153)
16 A iOS                      0x0000000100fe6a18 boost::asio::io_service::run() (io_service.ipp:59)
17 A iOS                      0x0000000100f9557c unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::call<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&, void const*) const (mem_fn_template.hpp:40)
18 A iOS                      0x0000000100f954b4 unsigned long boost::_mfi::mf0<unsigned long, boost::asio::io_service>::operator()<boost::shared_ptr<boost::asio::io_service> >(boost::shared_ptr<boost::asio::io_service>&) const (mem_fn_template.hpp:55)
19 A iOS                      0x0000000100f95460 unsigned long boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > >::operator()<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list0>(boost::_bi::type<unsigned long>, boost::_mfi::mf0<unsigned long, boost::asio::io_service>&, boost::_bi::list0&, long) (bind.hpp:243)
20 A iOS                      0x0000000100f953f4 boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > >::operator()() (bind_template.hpp:20)
21 A iOS                      0x0000000100f9539c boost::detail::thread_data<boost::_bi::bind_t<unsigned long, boost::_mfi::mf0<unsigned long, boost::asio::io_service>, boost::_bi::list1<boost::_bi::value<boost::shared_ptr<boost::asio::io_service> > > > >::run() (thread.hpp:117)
22 A iOS                      0x00000001007dac3c boost::(anonymous namespace)::thread_proxy(void*) (shared_ptr.hpp:668)
23 libsystem_pthread.dylib        0x00000001974efdc8 _pthread_body + 164
24 libsystem_pthread.dylib        0x00000001974efd24 _pthread_body + 158
25 libsystem_pthread.dylib        0x00000001974ecef8 thread_start + 4


Code: Select all

Thread : Crashed: Thread
0  libsystem_kernel.dylib         0x00000001971e7270 __pthread_kill + 8
1  libsystem_pthread.dylib        0x0000000197285170 pthread_kill + 112
2  libsystem_c.dylib              0x000000019715eb18 abort + 112
3  libc++abi.dylib                0x00000001961f1418 __cxa_bad_cast
4  libc++abi.dylib                0x0000000196210b8c default_unexpected_handler()
5  libobjc.A.dylib                0x0000000196a243dc _objc_terminate() + 156
6  libc++abi.dylib                0x000000019620dbb4 std::__terminate(void (*)()) + 16
7  libc++abi.dylib                0x000000019620d478 __cxxabiv1::exception_cleanup_func(_Unwind_Reason_Code, _Unwind_Exception*)
8  A iOS                      0x0000000100ecbc00 Sfs2X::Core::SFSProtocolCodec::DispatchRequest(boost::shared_ptr<Sfs2X::Entities::Data::ISFSObject>) (SFSProtocolCodec.cpp:186)
9  A iOS                      0x0000000100ecc818 Sfs2X::Core::SFSProtocolCodec::OnPacketRead(boost::shared_ptr<Sfs2X::Util::ByteArray>) (SFSProtocolCodec.cpp:93)
10 A iOS                      0x0000000100e1bcd4 Sfs2X::Core::SFSIOHandler::HandlePacketData(boost::shared_ptr<Sfs2X::Util::ByteArray>) (SFSIOHandler.cpp:389)
11 A iOS                      0x0000000100e17d04 Sfs2X::Core::SFSIOHandler::OnDataRead(boost::shared_ptr<Sfs2X::Util::ByteArray>) (SFSIOHandler.cpp:151)
12 A iOS                      0x0000000100e97c3c Sfs2X::Bitswarm::BitSwarmClient::OnSocketData(unsigned long long, boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >) (BitSwarmClient.cpp:895)
13 A iOS                      0x0000000100f173fc Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >) (DelegateOneArgument.h:54)
14 A iOS                      0x0000000100f1469c Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, boost::shared_ptr<void>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, boost::shared_ptr<void> > > > >) (ThreadManager.cpp:177)
15 A iOS                      0x0000000100f14110 Sfs2X::Core::ThreadManager::InThread() (ThreadManager.cpp:93)
16 A iOS                      0x0000000100f19cc0 boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>::operator()(Sfs2X::Core::ThreadManager*) const (mem_fn_template.hpp:49)
17 A iOS                      0x0000000100f19c30 void boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> >::operator()<boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>&, boost::_bi::list0&, int) (bind.hpp:253)
18 A iOS                      0x0000000100f19b98 boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > >::operator()() (bind_template.hpp:20)
19 A iOS                      0x0000000100f18f1c boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > > >::run() (thread.hpp:117)
20 A iOS                      0x00000001006fa318 boost::(anonymous namespace)::thread_proxy(void*) (shared_ptr.hpp:668)
21 libsystem_pthread.dylib        0x0000000197283dc8 _pthread_body + 164
22 libsystem_pthread.dylib        0x0000000197283d24 _pthread_body


Code: Select all

Thread : Crashed: Thread
0  libsystem_kernel.dylib         0x0000000194a1f270 __pthread_kill + 8
1  libsystem_pthread.dylib        0x0000000194abd170 pthread_kill + 112
2  libsystem_c.dylib              0x0000000194996b18 abort + 112
3  libsystem_c.dylib              0x000000019496ba3c basename + 90
4  A iOS                      0x0000000100fa71b4 boost::shared_ptr<Sfs2X::Util::EventDispatcher>::operator->() const (shared_ptr.hpp:653)
5  A iOS                      0x0000000100fa1470 Sfs2X::SmartFox::DispatchEvent(boost::shared_ptr<Sfs2X::Core::BaseEvent>) (SmartFox.cpp:1498)
6  A iOS                      0x0000000100fa1e40 Sfs2X::SmartFox::HandleConnectionProblem(boost::shared_ptr<Sfs2X::Core::BaseEvent>) (SmartFox.cpp:1448)
7  A iOS                      0x0000000100f9c8e8 Sfs2X::SmartFox::OnSocketIOError(unsigned long long, boost::shared_ptr<Sfs2X::Core::BaseEvent>) (SmartFox.cpp:1183)
8  A iOS                      0x0000000100feefd0 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<Sfs2X::Core::BaseEvent> >::Invoke(boost::shared_ptr<Sfs2X::Core::BaseEvent>) (DelegateOneArgument.h:54)
9  A iOS                      0x0000000100fee974 Sfs2X::Util::EventDispatcher::DispatchEvent(boost::shared_ptr<Sfs2X::Core::BaseEvent>) (EventDispatcher.cpp:71)
10 A iOS                      0x0000000100f53edc Sfs2X::Bitswarm::BitSwarmClient::DispatchEvent(boost::shared_ptr<Sfs2X::Bitswarm::BitSwarmEvent>) (BitSwarmClient.cpp:1011)
11 A iOS                      0x0000000100f515dc Sfs2X::Bitswarm::BitSwarmClient::OnSocketError(unsigned long long, boost::shared_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, Sfs2X::Core::Sockets::SocketErrors) (BitSwarmClient.cpp:973)
12 A iOS                      0x0000000100f6a448 Sfs2X::Util::DelegateTwoArguments<boost::shared_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, Sfs2X::Core::Sockets::SocketErrors>::Invoke(boost::shared_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, Sfs2X::Core::Sockets::SocketErrors) (DelegateTwoArguments.h:55)
13 A iOS                      0x0000000100f67be0 Sfs2X::Core::Sockets::TCPSocketLayer::CallOnError(boost::shared_ptr<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, Sfs2X::Core::Sockets::SocketErrors) (TCPSocketLayer.cpp:250)
14 A iOS                      0x0000000100f67734 Sfs2X::Core::Sockets::TCPSocketLayer::HandleErrorCallback(unsigned long long, boost::shared_ptr<void>) (TCPSocketLayer.cpp:223)
15 A iOS                      0x0000000100fcf840 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<void> >::Invoke(boost::shared_ptr<void>) (DelegateOneArgument.h:54)
16 A iOS                      0x0000000100fccaac Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, boost::shared_ptr<void>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, boost::shared_ptr<void> > > > >) (ThreadManager.cpp:186)
17 A iOS                      0x0000000100fcc3f4 Sfs2X::Core::ThreadManager::InThread() (ThreadManager.cpp:93)
18 A iOS                      0x0000000100fd1fa4 boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>::operator()(Sfs2X::Core::ThreadManager*) const (mem_fn_template.hpp:49)
19 A iOS                      0x0000000100fd1f14 void boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> >::operator()<boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>&, boost::_bi::list0&, int) (bind.hpp:253)
20 A iOS                      0x0000000100fd1e7c boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > >::operator()() (bind_template.hpp:20)
21 A iOS                      0x0000000100fd1200 boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > > >::run() (thread.hpp:117)
22 A iOS                      0x00000001007b0f38 boost::(anonymous namespace)::thread_proxy(void*) (shared_ptr.hpp:668)
23 libsystem_pthread.dylib        0x0000000194abbdc8 _pthread_body + 164
24 libsystem_pthread.dylib        0x0000000194abbd24 _pthread_body + 158
25 libsystem_pthread.dylib        0x0000000194ab8ef8 thread_start + 4


Can someone help us take a look at these and provide any direction on either what we're doing wrong or patches for these issues?

Thanks.
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: Crash reports

Postby Lapo » 16 Sep 2015, 13:49

Hi,
version 1.1.6 is quite old, I recommend downloading the latest API here:
http://smartfoxserver.com/download/sfs2x#p=client

Thanks
Lapo
--
gotoAndPlay()
...addicted to flash games
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 01 Oct 2015, 13:06

Ok we've finished upgrading to the latest 1.6.1 and added the deadlock fix noted in another thread. We are still seeing the same crashes. We get live crash reports for our live games. These crashes (smart fox C++ related ones) are the most frequent crashes that currently occur in all versions of our games for all the different platforms that it is available on. We've tried walking through the source code of these but aren't sure how to fully resolve these issues. Please help us look into these as soon as possible. As I mentioned before, these are the most frequent crashes that are occurring in our games at the moment.

Thanks.

We are still seeing the following crashes:

This one happens frequently and seemingly randomly on all types of Android devices.
Crash 1:
Thread : Crashed: app
SIGSEGV SEGV_ACCERR at 0x8000000000000000
0 ??? 0xb7f8cc18
1 libapp.so 0xa1feb728 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > >)
2 libapp.so 0xa2015d24 Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::map<std::string, boost::shared_ptr<void>, std::less<std::string>, std::allocator<std::pair<std::string const, boost::shared_ptr<void> > > > >)
3 libapp.so 0xa2015328 Sfs2X::Core::ThreadManager::InThread()
4 libapp.so 0xa2019b64 boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>::operator()(Sfs2X::Core::ThreadManager*) const
5 libapp.so 0xa2019a90 void boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> >::operator()<boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>&, boost::_bi::list0&, int)
6 libapp.so 0xa201994c boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > >::operator()()
7 libapp.so 0xa2019740 boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > > >::run()
8 libapp.so 0xa26b4d15
9 libc.so 0xb6e3334f
10 libc.so 0xb6e31367

This next crash occurs sporadically on iOS devices - and it happens if we call SmartFox::Disconnect(). It's hard to reproduce, but does happen sometimes when we ask SmartFox to Disconnect() after logging out (calling Disconnect() from OnLogout() on successful logout)
Crash 2:
Thread : Crashed: Thread
SIGABRT ABORT at 0x000000019b11f1e0
0 libsystem_kernel.dylib 0x000000019b11f1e0 __pthread_kill + 8
1 libsystem_pthread.dylib 0x000000019b1e8f0c pthread_kill + 112
2 libsystem_c.dylib 0x000000019b092b78 abort + 140
3 libsystem_malloc.dylib 0x000000019b14c384 szone_size + 522
4 App iOS 0x000000010001f270 boost::detail::shared_count::~shared_count() (sp_counted_base_sync.hpp:128)
5 App iOS 0x000000010072dd44 Sfs2X::Protocol::Serialization::DefaultSFSDataSerializer::DecodeObject(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5408884
6 App iOS 0x000000010072c738 Sfs2X::Protocol::Serialization::DefaultSFSDataSerializer::DecodeSFSObject(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5403240
7 App iOS 0x000000010072c49c Sfs2X::Protocol::Serialization::DefaultSFSDataSerializer::Binary2Object(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5402572
8 App iOS 0x00000001006eddf8 Sfs2X::Entities::Data::SFSObject::NewFromBinaryData(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5146920
9 App iOS 0x000000010076eec4 Sfs2X::Core::SFSProtocolCodec::OnPacketRead(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5675508
10 App iOS 0x0000000100725628 Sfs2X::Core::SFSIOHandler::HandlePacketData(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5374296
11 App iOS 0x0000000100722c00 Sfs2X::Core::SFSIOHandler::OnDataRead(boost::shared_ptr<Sfs2X::Util::ByteArray>) + 5363504
12 App iOS 0x0000000100759748 Sfs2X::Bitswarm::BitSwarmClient::OnSocketData(unsigned long long, boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >) + 5587576
13 App iOS 0x000000010078b5a4 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::__1::vector<unsigned char, std::__1::allocator<unsigned char> > >) + 5791956
14 App iOS 0x000000010078a29c Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::__1::map<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, boost::shared_ptr<void>, std::__1::less<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::allocator<std::__1::pair<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const, boost::shared_ptr<void> > > > >) + 5787084
15 App iOS 0x000000010078a05c Sfs2X::Core::ThreadManager::InThread() + 5786508
16 App iOS 0x0000000100204030 boost::(anonymous namespace)::thread_proxy(void*) (shared_ptr.hpp:668)
17 libsystem_pthread.dylib 0x000000019b1e7b3c _pthread_body + 156
18 libsystem_pthread.dylib 0x000000019b1e7aa0 _pthread_body + 154
19 libsystem_pthread.dylib 0x000000019b1e5030 thread_start + 4

This crash occurs on iOS as well. It happens when we put the app into the background and bring it back to the foreground. When bringing the app back into foreground, it may crash with the following crash report. This doesn't happen every time either, but it does happen sporadically.
Crash 3:
Thread : Crashed: Thread
EXC_BAD_ACCESS KERN_INVALID_ADDRESS at 0x0000000000000088
0 App iOS 0x000000010078a12c boost::asio::detail::kqueue_reactor::start_op(int, int, boost::asio::detail::kqueue_reactor::descriptor_state*&, boost::asio::detail::reactor_op*, bool, bool) + 5655644
1 App iOS 0x0000000100789eec boost::asio::detail::reactive_socket_service_base::start_op(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, int, boost::asio::detail::reactor_op*, bool, bool, bool) + 5655068
2 App iOS 0x0000000100789de4 void boost::asio::detail::reactive_socket_service_base::async_send<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::detail::reactive_socket_service_base::base_implementation_type&, boost::asio::mutable_buffers_1 const&, int, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&) + 5654804
3 App iOS 0x0000000100789d1c boost::asio::async_result<boost::asio::handler_type<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, void (boost::system::error_code, unsigned long)>::type>::type boost::asio::stream_socket_service<boost::asio::ip::tcp>::async_send<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >(boost::asio::detail::reactive_socket_service<boost::asio::ip::tcp>::implementation_type&, boost::asio::mutable_buffers_1 const&, int, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >&&) + 5654604
4 App iOS 0x00000001007892b4 Sfs2X::Core::Sockets::TCPClient::OnBoostAsioDataSent(boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&) + 5651940
5 App iOS 0x000000010078ab20 boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>::operator()(Sfs2X::Core::Sockets::TCPClient*, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&) const + 5658192
6 App iOS 0x000000010078aa8c void boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()>::operator()<boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list2<boost::system::error_code const&, unsigned long const&> >(boost::_bi::type<void>, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>&, boost::_bi::list2<boost::system::error_code const&, unsigned long const&>&, int) + 5658044
7 App iOS 0x000000010078aa28 void boost::asio::asio_handler_invoke<boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long> >(boost::asio::detail::binder2<boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> >, boost::system::error_code, unsigned long>&, ...) + 5657944
8 App iOS 0x000000010078a874 boost::asio::detail::reactive_socket_send_op<boost::asio::mutable_buffers_1, boost::_bi::bind_t<void, boost::_mfi::mf4<void, Sfs2X::Core::Sockets::TCPClient, boost::shared_ptr<unsigned char>, unsigned long, unsigned long, boost::system::error_code const&>, boost::_bi::list5<boost::_bi::value<Sfs2X::Core::Sockets::TCPClient*>, boost::_bi::value<boost::shared_ptr<unsigned char> >, boost::_bi::value<unsigned long>, boost::arg<2> (*)(), boost::arg<1> (*)()> > >::do_complete(boost::asio::detail::task_io_service*, boost::asio::detail::task_io_service_operation*, boost::system::error_code const&, unsigned long) + 5657508
9 App iOS 0x00000001007a5680 boost::asio::detail::task_io_service::do_run_one(boost::asio::detail::scoped_lock<boost::asio::detail::posix_mutex>&, boost::asio::detail::task_io_service_thread_info&, boost::system::error_code const&) + 5767600
10 App iOS 0x00000001007a530c boost::asio::detail::task_io_service::run(boost::system::error_code&) + 5766716
11 App iOS 0x0000000100783be0 boost::asio::io_service::run() + 5629712
12 App iOS 0x0000000100224030 boost::(anonymous namespace)::thread_proxy(void*) (shared_ptr.hpp:668)
13 libsystem_pthread.dylib 0x0000000193cebdc8 _pthread_body + 164
14 libsystem_pthread.dylib 0x0000000193cebd24 _pthread_body + 158
15 libsystem_pthread.dylib 0x0000000193ce8ef8 thread_start + 4
Last edited by dhuang11 on 12 Oct 2015, 07:23, edited 1 time in total.
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 05 Oct 2015, 10:05

Hi -

Any updates on these issues? For the first one we see on Android, it appears to happen when the player has been connected for a long time.
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: Crash reports

Postby Lapo » 05 Oct 2015, 10:36

Our C++ developer is taking a look. Stay tuned...
thanks
Lapo

--

gotoAndPlay()

...addicted to flash games
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 06 Oct 2015, 08:48

Hi Lapo,

Thanks for the response and follow-up. Just checking back again. The first crash listed is causing us the most problems (accounts for nearly 70% of all reported crashes):

Thread : Crashed:
SIGSEGV SEGV_ACCERR at 0x8000000000000000
0 ??? 0x9165d1b0
1 p.so 0x99b60908 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > >)
2 p.so 0x99b9fd50 Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::map<std::string, boost::shared_ptr<void>, std::less<std::string>, std::allocator<std::pair<std::string const, boost::shared_ptr<void> > > > >)
3 p.so 0x99b9f354 Sfs2X::Core::ThreadManager::InThread()
4 p.so 0x99ba3bb4 boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>::operator()(Sfs2X::Core::ThreadManager*) const
5 p.so 0x99ba3ae0 void boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> >::operator()<boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list0>(boost::_bi::type<void>, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>&, boost::_bi::list0&, int)
6 p.so 0x99ba399c boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > >::operator()()
7 p.so 0x99ba3790 boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf0<void, Sfs2X::Core::ThreadManager>, boost::_bi::list1<boost::_bi::value<Sfs2X::Core::ThreadManager*> > > >::run()
8 p.so 0x9a2463a5
9 libc.so 0xb6e9bbbb
10 libc.so 0xb6e99c83

Please let us know if there's anything additional we can provide to you to help resolve this.

Thanks!
User avatar
Lapo
Site Admin
Posts: 22999
Joined: 21 Mar 2005, 09:50
Location: Italy

Re: Crash reports

Postby Lapo » 07 Oct 2015, 08:41

Thanks. We're taking a look, at the moment we haven't been able to reproduce any of these issues.
We're doing a few more tests in the next days and we'll let you know.

cheers
Lapo

--

gotoAndPlay()

...addicted to flash games
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 07 Oct 2015, 12:58

Ok - we think we may know what's wrong - but this is a bit hard to reproduce.

In our game, there is a mechanism that disconnects the player if they repeatedly receive a slow ping to the SFS2X server (we set a threshold of continued slow or missed pings). This helps ensure that all players have a good play experience and Player A's lag doesn't impact the experience of Player B. We think what's causing this issue is a thread safety issue with the ThreadManager.cpp class.

Here are the 2 ThreadManager methods we think are causing a thread safety issue:

Code: Select all


ThreadManager::~ThreadManager()
{
   running = false;
   if (this->inThread != NULL)
   {
      this->inThread->interrupt();
      this->inThread->join();
      this->inThread = boost::shared_ptr<boost::thread>();
   }
   if (this->outThread != NULL)
   {
      this->outThread->interrupt();
      this->outThread->join();
      this->outThread= boost::shared_ptr<boost::thread>();
   }

   boost::shared_ptr<map<string, boost::shared_ptr<void> > > item;

   while (inThreadQueue->size() > 0)
   {
      item = inThreadQueue->front();
      inThreadQueue->pop_front();

      item->clear();
   }

   while (outThreadQueue->size() > 0)
   {
      item = outThreadQueue->front();
      outThreadQueue->pop_front();

      item->clear();
   }
}

// -------------------------------------------------------------------
// InThread
// -------------------------------------------------------------------
void ThreadManager::InThread()
{
    boost::posix_time::milliseconds workTime(5);
   
    while (running)
    {
        boost::this_thread::sleep(workTime);
       
        if (running == false) break;
       
        boost::shared_ptr<map<string, boost::shared_ptr<void> > > item;
       
        while (true)
        {
            inQueueLocker.lock();
            if(inThreadQueue->size() <= 0) {
                inQueueLocker.unlock();
                break;
            }
            item = inThreadQueue->front();
            inThreadQueue->pop_front();
            inQueueLocker.unlock();
           
            ProcessItem(item);
            item->clear();
        }
    }
}


During a disconnect, ThreadManager I believe is also shut down and the destructor may get called. In the ThreadManager destructor, it first interrupts the inThread and outThread which are both working to deplete the queue. However, whatever is already being done in inThread or outThread will still attempt to complete. As the destructor works through clearing items, the last cycle in the inThread. Beyond that, if you recall a fix we had submitted (and MBagnati approved our code change as a viable fix) to prevent deadlocking in the inThread() and outThread() functions, it actually lengthened the amount of time the inThread can spend trying to deplete it's queue (see now that there are 2 loops - while(running) and then another while(true) inside the while(running). Based on what we're seeing here, it looks like it's possible for the thread deconstructor to run and attempt to clear out an item, while at the same time, the InThread (or OutThread) attempts to consume the same item from a different thread.

Also, it might be that you cannot reproduce this specific issue because you guys are trying to reproduce on the 1.6.1 which does not yet include the approved deadlock fixes we previously proposed which makes the InThread and OutThread functions work differently from what you're seeing in order to prevent deadlocking. Does this theory make sense? What are your thoughts?
MBagnati
Posts: 126
Joined: 12 Feb 2013, 10:57

Re: Crash reports

Postby MBagnati » 07 Oct 2015, 23:14

Hi,
I try to add some notes & hypotheses about "Crash 1" that occurs in ThreadManager::ProcessItem.
  • To reproduce the crash, I am working with API rel. 1.6.1 updated with deadlock's fix described in http://www.smartfoxserver.com/forums/viewtopic.php?f=34&t=18054
    I confirm that deadlock's fix, that you have suggested in the past, is good.
  • I hypothesize that the reason why I can not reproduce the crash is due to the fact that I am not recreating the scenario that occurs in the game.
  • In my opinion there are two paths that are good candidates to be investigated:

    first: as you suggest, may be that disconnection from SmartFoxServer causes a conflict in ThreadManager class between a thread that executes the class destructor where queue and queue's items are destroyed and another thread that is processing items contained into the queue.

    To avoid this conflict, there is the this->inThread->join() call inside the ThreadManager destructor

    So, I hope that ThreadManager destructor cannot invalidate queued items while ThreadManager thread is running.
    Boost documentation about join() function says: " join() will block the calling thread until the thread represented by the boost::thread object has completed."


    second: looking at the call stack that you have provided:

    Code: Select all

          1 libapp.so 0xa1feb728 Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > >)
          2 libapp.so 0xa2015d24 Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::map<std::string, boost::shared_ptr<void>, std::less<std::string>, std::allocator<std::pair<std::string const, boost::shared_ptr<void> > > > >)
          3 libapp.so 0xa2015328 Sfs2X::Core::ThreadManager::InThread()

    may be that problem is not the corrupted state of an item extracted from the queue, but may be the callback that ThreadManager::ProcessItem
    through Util::DelegateOneArgument wants to invoke.

    We know (because we have the call stack that shows the execution of ThreadManager::ProcessItem) that ThreadManager thread is running.
    It is not neither stopped nor destroyed.
    Thread procedure ThreadManager::InThread is processing an item retrieved from the queue.
    As I have said above, if another thread is calling the ThreadManager destructor, I think that it is suspended by this->inThread->join() and has not invalidated
    queued items.

    So, may be that crash is generated by ThreadManager::ProcessItem that is processing a just de-queued item and
    wants to call a callback that no more exists.

    Would be interesting to reproduce the crash while API is running under debugger to watch in ThreadManager::ProcessItem the value
    of "data" variable at the statment onData->Invoke(data) in that moment.
    Probably "data" contains the name of a callback that belongs to an object that has been destroyed

    So, my hypothesis is:

    ThreadManager thread is running
    The game is exchanging data with SmartFoxServer so inThreadQueue & outThreadQueue of ThreadManager are in use.
    ThreadManager::ProcessItem is processing input items.
    ThreadManager::ProcessItem is calling the callback designed for data incoming from socket (generally BitSwarmClient::OnSocketData).
    Socket goes in disconnected/error/destroyed state
    An item with a bad callback value is enqueued into inThreadQueue: the processing of this item will cause the crash

    Please, can you verify if BitSwarmClient::Dispose() is called in your game by mechanism that disconnects the player if they repeatedly receive a
    slow ping ?
    If called can you perform a test? Can you edit BitSwarmClient::Dispose method to move the
    this->threadManager = boost::shared_ptr<Sfs2X::Core::ThreadManager>();
    from current position to the begin of the routine?

    Code: Select all

    void BitSwarmClient::Dispose()
    {
       isDisposed = true;

       this->threadManager = boost::shared_ptr<Sfs2X::Core::ThreadManager>();                           // MOVED HERE

       if (socket != NULL)
       {
          lockDispose.lock();

          boost::shared_ptr<ConnectionDelegate> delegateConnection = boost::shared_ptr<ConnectionDelegate>();
          socket->OnConnect(delegateConnection);

          boost::shared_ptr<ConnectionDelegate> delegateDisconnection = boost::shared_ptr<ConnectionDelegate>();
          socket->OnDisconnect(delegateDisconnection);

          boost::shared_ptr<OnDataDelegate> delegateOnData = boost::shared_ptr<OnDataDelegate>();
          socket->OnData(delegateOnData);

          boost::shared_ptr<OnErrorDelegate> delegateOnError = boost::shared_ptr<OnErrorDelegate>();
          socket->OnError(delegateOnError);

          lockDispose.unlock();
       }

       this->sfs = boost::shared_ptr<SmartFox>();

    //   this->threadManager = boost::shared_ptr<Sfs2X::Core::ThreadManager>();                        // MOVED FROM HERE
cheers
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 08 Oct 2015, 06:01

Hi-

Thanks for the thorough and insightful response. So a few points and even more questions.

1. We don't call SmartFox::Dispose() right after a Disconnect(). We call Dispose() if later on in the game flow, the player decides to reconnect to our game server. We don't call SmartFox::Dispose() right after calling SmartFox::Disconnect() because we've noticed that Dispose() will crash if called too quickly after a SmartFox::Disconnect(). However, if we call it later (e.g. if the Player decides to reconnect to server - before making a new connection) Dispose() succeeds with no issues. We've tried different ways to call Dispose() around the actual Disconnect() and have found that it always crashes for us. It crashes in the ThreadManager::~ThreadManager() destructor -

2. We've noticed that in SmartFox::HandleClientDisconnection() and SmartFox::OnSocketClose() there is a Reset() that seems to dispose half of the data used by the SmartFox class. The remaining is cleared in Dispose(). Is it possible that if we delay the SmartFox::Reset() call and lump it together with Dispose() that we wouldn't see this crash? Basically leave the SmartFox and related objects/memory intact until we explicitly Dispose?

3. What happens if the disconnection isn't initiated from the client? Say the socket is disconnected because the internet has cut out? How/when does SmartFox::Dispose() get called in that situation? We looked through this - it doesn't seem to get called in that scenario. However, we noticed that SmartFox::Reset() will get called and the ON_CONNECTION_LOST event will get fired as well. Should we always do a SmartFox::Dispose() when handling ON_CONNECTION_LOST Events?

Also on a separate note, for crashes #2 and #3 on iOS. We have resolved them. It's not anything in the SmartFox code that had to be changed but a usage pattern that we utilized. In general, we utilize iOS background tasks to keep SmartFox connections alive and receiving/sending data even if the app is in the background. iOS background tasks are allotted a short duration of time before iOS essentially stops/pauses all app background threads. Previously, we would try to logout the user before disconnecting. If this took too long for the server to come back, iOS would end up killing all threads which would trigger another disconnect because the socket connection is lost. To resolve those issues, we prevented double/multiple calls into SmartFox disconnects.
MBagnati
Posts: 126
Joined: 12 Feb 2013, 10:57

Re: Crash reports

Postby MBagnati » 09 Oct 2015, 23:21

Hi,
I have done an intensive test session to try to reproduce the crash #1.
I have emulated an high-volume data exchange...random disconnections required from client...random TcpIp fall down...etc.etc...
Unfortunately all tests have returned a positive feedback, I have not seen any crash :-(
I have also followed a different approach: the "reading" of source code around ThreadManager::ProcessItem to search potentially faults.
From this activity, I have discovered a very particular scenario that causes a crash.
There is an API bug when two notifications regarding the TcpIp connection between Client API and SmartFox Server are queued in ThreadManager
May be that the processing of first notification disposes the socket, so the process of second one will go in a crash condition.
I do not know if this is what happens in your game...but this is the only fault that I have found.
I have changed four files to fix the problem:

Core\ThreadManager.h
Core\ThreadManager.cpp
Util\DelegateOneArgument.h
BitSwarm\BitSwarmClient.cpp

I attach them to this message.
Please, can you perform a test overriding your API with these files?

Let me know if this fix solves the crash

About your question on Reset() and ON_CONNECTION_LOST, I am taking a look.

Thanks for your help
Massimo
Attachments
Changed Files.zip
(9.82 KiB) Downloaded 721 times
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 10 Oct 2015, 07:02

Hi Massimo -

Tried the changes you posted. It still crashes and in the same spot on Android for iOS, we've also seen a similar crash but it happens at a different point.

A few things to note, from what we've seen, there appears to be a couple ways that get to the same crash. Is it possible that this isn't Disconnect related? We see a crash in DelegateOneArgument that starts processing from ThreadManager::ProcessItem(), are there other possibilities aside from an actual disconnection of some sort that can cause this crash? Also, do you have an Android device? We can share with you the app APK, steps to reproduce this crash (it takes a little bit of time - but it will crash and seemingly faster on specific device makes/models) and also the debug symbols so you can step through what we're seeing. We've also made this crashlog available to be viewed on our live crash reporting tool you can view it here http://crashes.to/s/cbe1c79afb4

For iOS we see a very similar looking crash that we can reproduce by forcing a disconnect of SmartFox while the app is running in the background. At that point on iOS, we can see that after everything has been disconnected (SmartFox::IsConnected() == false), that the ThreadManager is still running and tries to call FnUserCountChange(). In this scenario, it crashes in FnUserCountChange. For this scenario, we've been able to resolve this crash by adding the following code:

Code: Select all

void ThreadManager::Stop() {
    running = false;
}

void BitSwarmClient::Disconnect() {
    if(threadManager) {
        threadManager->Stop();
    }

    ....
}

void BitSwarmClient::Destroy() {
    if(threadManager) {
        threadManager->Stop();
    }

    ....
}

void BitSwarmClient::ReleaseResources() {
    if(threadManager) {
        threadManager->Stop();
    }

    ....
}

void BitSwarmClient::OnSocketError(unsigned long long context, boost::shared_ptr<string> message, SocketErrors se) {
    ...
       if (se != SocketErrors_AccessDenied)
   {
      if (!instance->attemptingReconnection && !instance->sfs->IsConnecting() && !instance->manualDisconnection)
      {
                    if(instance->threadManager) {
                        instance->threadManager->Stop();
                    }
                    ...
      }
      ...
   }
}

void ThreadManager::ProcessOutItem(boost::shared_ptr<map<string, boost::shared_ptr<void> > > item)
{
    if(!running) {
        return;
    }
    ....
}

void ThreadManager::ProcessItem(boost::shared_ptr<map<string, boost::shared_ptr<void> > > item)
{
    if(!running) {
        return;
    }
    ...
}




Daniel
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 11 Oct 2015, 10:21

One other observation - this crash happens a lot more frequently when we use release builds. NDK_DEBUG = 0, which also sets the optimization flag to -O2.
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 12 Oct 2015, 06:42

Also - from Android NDK

Code: Select all

10-12 14:39:27.362: I/DEBUG(25044): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
10-12 14:39:27.362: I/DEBUG(25044): Build fingerprint: 'google/occam/mako:4.2.2/JDQ39/573038:user/release-keys'
10-12 14:39:27.362: I/DEBUG(25044): Revision: '11'
10-12 14:39:27.362: I/DEBUG(25044): pid: 6922, tid: 7056, name: Thread-5532  >>> poker <<<
10-12 14:39:27.362: I/DEBUG(25044): signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 72b12940
10-12 14:39:27.522: I/DEBUG(25044):     r0 00000000  r1 00000000  r2 79176d98  r3 79176d98
10-12 14:39:27.522: I/DEBUG(25044):     r4 00000000  r5 00000000  r6 72b12940  r7 72b27160
10-12 14:39:27.522: I/DEBUG(25044):     r8 769b3885  r9 00100000  sl 72b27160  fp 79176db4
10-12 14:39:27.522: I/DEBUG(25044):     ip 00000000  sp 79176d90  lr 762ccffc  pc 72b12940  cpsr 60000010
10-12 14:39:27.522: I/DEBUG(25044):     d0  0000000000000014  d1  6c61632065707954
10-12 14:39:27.522: I/DEBUG(25044):     d2  2073756f69766579  d3  30203a6574617470
10-12 14:39:27.522: I/DEBUG(25044):     d4  0300122a00800002  d5  0200030012700100
10-12 14:39:27.522: I/DEBUG(25044):     d6  7201000000036373  d7  01766a0008000004
10-12 14:39:27.522: I/DEBUG(25044):     d8  0000000000000000  d9  0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d10 0000000000000000  d11 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d12 0000000000000000  d13 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d14 0000000000000000  d15 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d16 417766a008000000  d17 3fe0000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d18 0000000000000000  d19 3fe0000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d20 3fe0000000265699  d21 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d22 0000000000000000  d23 bf66c16c16bebd93
10-12 14:39:27.522: I/DEBUG(25044):     d24 3fc555555555553e  d25 3ff0000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d26 3fd55555518f264d  d27 3fdb6db6db6fabff
10-12 14:39:27.522: I/DEBUG(25044):     d28 3fe3333333333303  d29 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     d30 0000000000000004  d31 0000000000000000
10-12 14:39:27.522: I/DEBUG(25044):     scr 6000009a
10-12 14:39:27.522: I/DEBUG(25044): backtrace:
10-12 14:39:27.522: I/DEBUG(25044):     #00  pc 0002a940  <unknown>
10-12 14:39:27.532: I/DEBUG(25044):     #01  pc 00c2bff8  /data/app-lib/poker-1/libpoker.so (Sfs2X::Util::DelegateOneArgument<boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > > >::Invoke(boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > >)+68)
10-12 14:39:27.532: I/DEBUG(25044): stack:
10-12 14:39:27.532: I/DEBUG(25044):          79176d50  769b3885  /data/app-lib/poker-1/libpoker.so
10-12 14:39:27.532: I/DEBUG(25044):          79176d54  76cce5f8  /data/app-lib/poker-1/libpoker.so
10-12 14:39:27.532: I/DEBUG(25044):          79176d58  79176d6c  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d5c  75e4015c  /data/app-lib/poker-1/libpoker.so (boost::detail::sp_counted_base::add_ref_copy()+32)
10-12 14:39:27.532: I/DEBUG(25044):          79176d60  70c19354 
10-12 14:39:27.532: I/DEBUG(25044):          79176d64  70c19350 
10-12 14:39:27.532: I/DEBUG(25044):          79176d68  79176d7c  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d6c  75e402e0  /data/app-lib/poker-1/libpoker.so (boost::detail::shared_count::shared_count(boost::detail::shared_count const&)+68)
10-12 14:39:27.532: I/DEBUG(25044):          79176d70  79176dfc  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d74  79176d9c  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d78  79176d8c  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d7c  762829e4  /data/app-lib/poker-1/libpoker.so (boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > >::shared_ptr(boost::shared_ptr<std::vector<unsigned char, std::allocator<unsigned char> > > const&)+64)
10-12 14:39:27.532: I/DEBUG(25044):          79176d80  79176df8  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d84  79176d98  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d88  df0027ad 
10-12 14:39:27.532: I/DEBUG(25044):          79176d8c  00000000 
10-12 14:39:27.532: I/DEBUG(25044):     #00  79176d90  79176df8  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          ........  ........
10-12 14:39:27.532: I/DEBUG(25044):     #01  79176d90  79176df8  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176d94  72845b48 
10-12 14:39:27.532: I/DEBUG(25044):          79176d98  733cc760 
10-12 14:39:27.532: I/DEBUG(25044):          79176d9c  70c19350 
10-12 14:39:27.532: I/DEBUG(25044):          79176da0  79176db4  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176da4  72845b48 
10-12 14:39:27.532: I/DEBUG(25044):          79176da8  76cce644  /data/app-lib/poker-1/libpoker.so
10-12 14:39:27.532: I/DEBUG(25044):          79176dac  769b3885  /data/app-lib/poker-1/libpoker.so
10-12 14:39:27.532: I/DEBUG(25044):          79176db0  79176e0c  [stack:7056]
10-12 14:39:27.532: I/DEBUG(25044):          79176db4  7630c5cc  /data/app-lib/poker-1/libpoker.so (Sfs2X::Core::ThreadManager::ProcessItem(boost::shared_ptr<std::map<std::string, boost::shared_ptr<void>, std::less<std::string>, std::allocator<std::pair<std::string const, boost::shared_ptr<void> > > > >)+944)
dhuang11
Posts: 38
Joined: 02 May 2014, 08:21

Re: Crash reports

Postby dhuang11 » 12 Oct 2015, 17:53

Just to provide an update on things we've tried:

1. We've noticed on iOS that not actually setting ThreadManager::running to false when disconnecting while app is in background will cause 2 of the 3 crashes listed above. By actually forcing a stop on ThreadManager as detailed in the other posts in this thread, this issue can no longer be reproduced. So I think the ThreadManager needs to be stopped when SmartFox::Disconnect() is called and before SmartFox::Reset() is run as part of the SmartFox::Disconnect() call stack.

The rest of these are about issue #1 on Android (the trickier issue)
1. Using the code changes you provided does not make the issue happen less frequently. We've been able to log/note that all crashes occur from EnqueueDataCall and not EnqueueCustom.
2. For the first issue on Android, we tried unwrapping boost::shared_ptr's and wrapping back up using C++11 std::shared_ptr in ThreadManager::EnqueueDataCall and unwrapping from std::shared_ptr to boost::shared_ptr before calling the ProcessItem seems to minimize the crash - at least we haven't run into it after testing it this way. The code isn't pretty, but it seems to indicate something wrong with boost::shared_ptr.
3. We were previously running with android:debuggable = true or set to false in release builds in AndroidManifest.xml and NDK_DEBUG=1 regardless of what android:debuggable is set to - this we can reproduce the 1st issue pretty easily. Note that when android:debuggable = false and NDK_DEBUG = 1, the issue seems to reproduce more frequently than if we run with android:debuggable = true and NDK_DEBUG=1
4. When we set NDK_DEBUG = 0 and android:debuggable=false and recompile everything, the issue becomes hard to reproduce. So far (5 hours in) we haven't been able to reproduce in this combination. The prior combination sets described in #3 above, we could reproduce within 10-15 minutes - sometimes as quickly as a few seconds on launching the app and connecting to SFS and playing the game. Note that we've compiled the boost libraries with GCC optimization flag -O3, and when we compile with NDK_DEBUG = 0, the default optimization is -O2.
5. We've tried changing ThreadManager::inThreadQueue to a vector instead of a list - this doesn't do anything - we were prompted to test this because we googled that someone else was having issues with std::queue (which is very similar to a std::list) and wrapping into boost::shared_ptrs that crash after several hours. It sounded a lot like the issue we were running into.
6. We've noticed that amount of free RAM on the device does not seem to impact reproducing of issue #1 - same for connection speed/quality - seems to be irrelevant
7. We've also been able to reproduce this issue thus far only on LG Nexus 4, Samsung Note 3, HTC 820, Sony D6653, Moto DROID RAZR, and a handful of other popular devices running any version of Android from 4.x to 5.x. We have not been able to reproduce successfully on most other device types. This seems to indicate that there's something processor dependent. We have not correlated processor types on these device models to see if there's something common about all these device types yet but intend to do so shortly.
8. Note running the ndk-gdb debugger is very unwieldy and also setting everything to be debuggable seems to make this issue very hard to reproduce (although still reproducible)

To conclude what we've seen so far - we've started to think that this issue is very compiler dependent and how a specific type of ARM processor in a large # of Android devices interprets the compiled code. We've walked through the SFS2X C++ API code over and over again and can't seem to identify anything that would cause what we're seeing for issue 1 aside from some missing handling of Disconnects() for iOS devices described above. This led us to believe that boost::shared_ptr was not behaving as intended and was causing memory corruption in some scenarios. Right now, it seems like the optimization we compile SFS2X with and the rest of our app with needs to match what we previously compiled the boost libraries with. Running with android:debuggable=true and NDK_DEBUG=0 makes the issue harder to reproduce most of the time (still reproducible), but running android:debuggable=false and NDK_DEBUG=0 seems to chug along just fine.

These are our next steps:
1. Continue to run with NDK_DEBUG=0 (GCC -O2) and android:debuggable=false for several more hours and see if we can no longer reproduce this issue - if we get past 10 hours, we can declare success. On the LG Nexus 4 we've been testing with, we've been able to reproduce within 10-15 minutes with certain debug/release/optimizer combinations as noted above.
2. If #1 fails, we'll continue running the hack we made to wrap/unwrap and wrap/unwrap the objects underneath boost::shared_ptr to std::shared_ptr when putting into ThreadManager::inThreadQueue. Again, if we can get past 10 hours without running into this crash, then we can declare this as a success.
3. If #1 and #2 fail, we'll recompile our boost library without inline optimizations and using -O0 compiler optimization to see if that makes it behave a bit more sane. Same thing - if doing that allows us to run 5-6 hours without reproducing this crash, then it would be a success
4. We're currently compiling everything with Android NDK r9d, we plan on trying to recompile boost, sfs2x and everything else using the latest Android NDK r10e - same success criteria as the above
5. Finally, we're tempted to upgrade Boost to a newer version (1.59.0). We've done it briefly (not fully tested) and ran everything. Everything does appear to run fine at first glance.
6. It's possible next step #1,2,3,4,5 are all successful. If that is the case, our first choice is run with next step 1 as this involves the least amount of changes/hacks and risk

Issues like this remind me why I have a largely hate relationship with C++ especially post-release of a product. It's 2015, everything should just compile and go but that never seems to be the case with these compilers.

If we're successful with any of the above, I'll report back. At minimum I hope this will help other people who intend on using SFS2X with Android devices.

What are your thoughts? Also, is there a way to re-organize the SFS2X code in this area so that it would be less compiler/boost sensitive?

Return to “SFS2X C++ API”

Who is online

Users browsing this forum: No registered users and 6 guests