How to troubleshoot / test / benchmark usb hub (Hubs are acting as bottle neck...Why??)

I am running a Ubuntu 14.04 LTS and some unusual serial hardware, which uses serial to USB converters.

I am unable to plug several slow devices (RS232 to USB 2.0 converter, and RS422 to USB 2.0 converter, and another USB 2.0 device) into a 3.0 hub without having my application crash due to lack of incoming messages. However if all the devices are plugged directly into the PC everything works. Could you please explain why this is happening? Please read on for details...

I bought a 3.0 USB hub made by Orico. It has 4 ports, and it is powered by the USB port it plugs into.

I connected 2 serial converters (with devices to talk to) and a single USB device to it. Here are the details:

• RS422 to USB (2.0) - configured as /dev/ttyUSB0 at 500,000 8N1.
• RS232 to USB (2.0) - configured as /dev/ttyUSB1 at 115200 8N1.
• USB 2.0 device (Labjack U3, which is a general purpose IO system... collects pressed buttons and the like...).

I have observed some very strange behavior that cost me two 14 hour days of turbo high stress until I realized the hub is the issue.

"Bad" Behavior of USB Hub that took some time to discover:

• when only the RS232 to USB was connected, the serial device I was communicating to was receiving all the data the PC was sending, but all the responses from the device were being dropped by the hub (or more accurately, were dropped as long as the hub was used).
• When two devices were connected (Labjack USB 2.0 Device + RS422 to USB 2.0) everything was working (messages from both devices were getting to PC at a good rate). If the RS232 to USB converter was connected while the the other two devices were already connected everything was still fine, and I could configure the RS232 port. However as soon as the serial device connected to the RS232 was turned on and started sending "messages", the application running on the PC would crash with an error indicating it was starved for messages from the RS422 device (that should be arriving via the USB hub). This would happen within 5 seconds of turning on (bringing online) the serial device that was connected to the RS232 to USB converter.

I was baffled for a while... until I removed the USB hub and connected all devices directly to the PC... then everything started to work like a charm.

Could you please expalain why this is happening? As well, could you please suggest some tools or a proceedure to help me pinpoint exactly what is going wrong with the USB network?

I need the hub to carry all the serial data that gets converted to USB messages to the PC without a bottleneck; and when we consider that the serial speeds of my RS232 and RS422 devices are so slow (compared to USB... not my troubleshooting speeds...) I would think that any hub will do the trick. Having discovered this issue, I am curious why a hub with speeds much faster than the devices connected to it is acting as a bottle neck.

Please be aware that the hub is not (likely to be) defective... It carries data just fine; until it encounters 2 serial converters and a USB device plugged in all at the same time, at which point it's speed (or perhaps voltage) crashes my robot.

I suspected a voltage drop was causing an issue, so I used a powered 2.0 USB 7 port hub and the results were the same! Application is starved for messages as soon as all 3 devices start sending messages.

If I plug the RS422 device direct into the PC, and the RS232 and Labjack USB devices into the USB hub (and then USB hub into PC), the Labjack U3 fails with an error.

Does this mean that a USB hub cannot pass the data passed to it by 2 serial devices and a single USB device at the rate at which the messages are arriving? This sounds really off.

If I can provide more info, please let me know. Thanks in advance.

The RS232 devices are usually FS devices, with USB protocol running at 12Mbps. When plugged into a hub, USB2 or USB3, the communication goes over special communication processor called "Transaction Translator", or "TT", which is embedded into every USB hub. There are two kinds of hubs, the ones that have only one TT for all ports, and the ones that have individual TT for each downstream port.

If you have several FS devices connected to a single-TT hub, the buffering resource will be starved, and FS bandwidth (12Mbps) will be shared between connected devices. To get full FS bandwidth over each downstream connection you need to have so-called "Multi-TT" hub.

Also, the handling of FS transactions over TT requires special approach on host side, called "split transactions".

Modern computers have native FS support through XHCI (USB 3.0) host controllers, so direct connection is handled naturally by XHCI driver. Older computers have the FS support inside their hardware, and muliplex ports to older OHCI or UHCI controllers, so they also have direct support for FS traffic rates.

However, if a hub is in between, the FS traffic must be handled by split-transaction technique, which is solely software-based. The split rules are horribly complicated, and covered by about 150 pages of USB specifications. If your robot driver has poor (or no) support for split transactions, FS and LS devices would fail to work behind a hub, specially if there are many of these devices.

Addition: Orico hub uses VL812 controller IC, which has only one TT in HS path. With (presumably FS) 422-USB device configured for 500,000 bauds and other FS devices, it likely has no bandwidth left, and host has problem with transaction scheduling and barfs on something.