I don't think that you can achieve low latency operation with a Virtual Box guest OS at all. That does not mean it is not technically feasible, it just seems that Virtual Box is not designed for that. In fact Apple has probably the best virtualization system available in their technology toolbox in regard to real-time stability, which really delivers rock solid <1ms latency, however Apple currently only uses it for isolating sandboxed Mac and iOS apps, not for installing guest OSs on your Mac.
Maybe one of the many other virtualization products on the market is able to deliver low latency. But you need to test that. I could not tell.
You are aware that both LinuxSampler and Jack are available as native binaries for Mac, right?