I spent some time looking into this issue. The connection loop isn't actually infinite, as far as I can tell. The issue is that each time a CallMatlabFunction command is initialized, it looks for an open MATLAB engine. When it doesn't find one, it tries to open one, and if there is a license issue, that attempt fails. Each instance of the command is initialized twice when the Sandbox is loaded – once in the Sandbox::Initialize() method, and once someplace else that I didn't get tracked down. So even simple scripts, like the sample script Ex_CallMatlabFunctions – which has 2 CallMatlabFunction commands – show multiple engine startup failures (4 for this sample script). Maybe we should add code to the MatlabInterface to count the startup failures (clearing the counter on success) and let the Sandbox access that counter to determine if it should quit the initialization process.
That approach seemed too risky to me for R2013a because it might produce unwanted side effects that we don't know to test for (at this writing we're already in the RC iteration cycles), but we ought to fix this issue early in the R2013b cycle, I think.