* Common race condition is fixed by identifying that Client.respHandler
can be completely removed since all respHandler operations (get, put
and invocation) can be moved into the Client.processLoop goroutine,
meaning that zero locking is required. This race condition resulted
in a deadlock that was resolved by the response timeout at the
end of client.Do, returning ErrLostConn
* Rare race condition is fixed by changing responseHandlerMap.get
to .getAndRemove. This race condition resulted in the innerHandler
for a new dtJobCreated assigned in client.Do overriding a stale older
dtJobCreated request, and the newer innerHandler being removed by an
older dtJobCreated in client.processLoop > client.handleInner. When
the newer dtJobCreated response was received, the handler for it had
already been deleted. This was resolved by the response timeout at the
end of client.Do, returning ErrLostConn
When receiving a response, what was happening
1. Read bufferSize and it gets assigned to leftdata
2. Read another bufferSize
3. 2 buffers get appended, but leftdata still points to first buffer
4. Process data buffer which contains only complete responses
5. Back to ReadLoop, but leftdata still points to first incomplete buffer
causing corrupt data to be processed
Solution is to make leftdata nil once we have merged it with the second buffer