You may want to contact Leopard Imaging for their 6 camera TX1/TX2 carrier card (LI-TX1-CB-6CAM).
That card has an FPGA for time tagging.
IMX185 has Vsync which can be connected to FPGA for accurate timestamp.
Video For Linux (V4L) also has timestamp with less precision than FPGA. Unfortunately V4L timestamp is broken for L4T 28.1.
Maybe a small follow up question: Could you explain what is broken in v4l and if libargus could solve this? E. G. Use Argus as an appsrc element for a gstreamer pipeline? What are your thoughts on this?