Share USB between Linux and CM3

Hi,

We have custom hardware based on RZN1D platform. We need to access the one USB from Linux (USB1/ Host) and another USB from CM3 (USB2 /Host). Currently we have enabled the USB2 from CM3 and disabled the same USB2 from Linux. During the next phase of development we want to enable the only USB1 from Linux as a Host. When we did this changes in DTB. We are only able to access USB2 from CM3 and Linux side we are getting kernel panic.

Is there any way we can enable one USB (USB1 / Host) from Linux and another USB (USB2 / Host) from CM3?? 

Regards,

Mayur