Inconsistent runtimes for chasers

[omegahacker]

I'm trying to build up lighting for a high-school drama production, and have been running into a range of serious problems. The topic of this particular post is the inaccuracy I'm seeing in the real runtime of chasers.

First, we have 3 moving heads, 2 pars, and 2 bar lights, nothing fancy. With the fixtures we have we're going to have to move the lights all over the place, both switching from one location to another, and following actors through scenes. As a musical (Beauty and the Beast), there are a lot of musical numbers that I want to actively lock down to specific times, because the music is already doing that. Others we'll have to step through manually.

I'm using a collection of scenes, some with just the movement channels for individual lights, others with the lighting channels, and then collecting the appropriate combination of scenes into a collection for each lighting cue, then listing those collections as steps in the chaser, with fade and duration set based on the music. (I'm brand new to QLC and this kind of lighting operation, so if that doesn't make sense I'd be happy to learn a better way - I'm going to start researching the Show Manager tonight, now that I've got a better handle on the actual drama cue...)

Here's the problem: for the first part of the prologue I put together a chaser with something like 10 lighting cues (collections of scenes per above) with a runtime of 1min40sec. As I was building the chaser, I kept having to adjust the times of the steps over and over, until I decided to time the entire chase. The first time I ran it I got 1min46sec from my phone's stopwatch. I restarted the program, and another run got closer to 1min41sec. Neither is sufficient, as even the smaller error would rapidly deviate far too much over the course of a 5-8min musical number.

I'm running on a laptop that should have more than enough power (I don't remember off the top of my head but I think it's at least an 8th gen i5), with updated Windows 10 and outputting over an Enttec Open DMX USB (that I've had for ~20 years....), and see no hint at all of strain on the machine, though I haven't looked at CPU load directly. However, I see that the countdown bar in the cue list tends to only get 3-5 "FPS", which seems rather odd considering the system should be running at the default 50Hz refresh. I also see stuttering in the light fades that seems to indicate that the effective DMX output rate might not be what it should.

I've set the enttecopendmxusb registry parameter to 256 as we're only at ~200 channels, with no obvious result. At one point I changed the mastertimer frequency to 100Hz, and the result was that the chaser time got *dramatically* worse, with the countdown clock running something like 1/2 or even 1/3 realtime, so that could be a hint.

I'm going to try to do some more experimentation tonight with a dummy show, but my guess is that the chaser is constantly doing a bunch of relative time delays, which give rise to a multitude of possibilities for unexpected delays caused by things like the Windows scheduler, rather than using absolute timings based on the computer's actual clock.

I need to find a solution to this inaccuracy, as I would very much prefer to be able to run each musical number in the show as a completely self-running chaser. There are going to be a *lot* of cues in this show, and having to do all of them by hand would be extremely error-prone... One potential workaround I can think of is to break the chaser into smaller groups, where a manual forward step triggers a group of 3-5 steps in the chaser that have specific timers attached, and the last one has an infinite duration that "resets" any accumulated timing errors, and hopefully occurs at an obvious and easy-to-hit cue in the production.

Are there any Windows-level optimisations that should be done? Another path forward may be running on a Linux machine with a realtime kernel? I'm a Linux guy anyway...

Thanks!

[GGGss]

1st - keep the default timings in place... do not tamper with the internal clock. This won't help at all. My only suggestion is to lower the DMX frequency in the interface's configuration. (in-output tab, universe with the open interface, press the config button. Lower the DMX frequency to 30Hz (or 24, or somewhere in between).
2nd—If you are in a time-critical situation, you must upgrade your DMX interface. With many fade times, your interface will struggle to keep pace. Your CPU is doing all the critical DMX calculations. If you have a 'pro' interface, this CPU load will be dissolved. Stuttering movements prove this. Every step in the fade equals a new strain of DMX calculations. As long as they are not confirmed, the QLC+ engine waits a bit - in your case, it has inconsistent timing.
3rd—Do not use the show component. QLC+ is not a timecode machine. Your timing with the show will surely be off with more than the desired offsets.
4th—If timing is critical, build in some manual 'go' steps inside the chaser. This will introduce an additional workload for the operator.
But I would start by lending a pro interface. Ask your local distributor if you may use it as a demo.

[janosvitok]

Another take: you may use another program (DAW) that can keep the time and that will send midi commands to QLC+ to trigger scenes.

[GGGss]

+1 for Jano

[kenact]

Have you updated the Enttec OpenDMX driver to the latest available? The latest available on the Enttec website is CDM v2.12.36.4 WHQL.

[omegahacker]

Have you updated the Enttec OpenDMX driver to the latest available? The latest available on the Enttec website is CDM v2.12.36.4 WHQL.

Yeah, I've been running that version directly from the FTDI site from the beginning.

I also hooked the DMX bus up to my digital scope (Rigol MSO4024) and confirmed that there are properly-formed DMX frames going out (almost) exactly every 33ms. The only glitch seems to be a particularly long MAB. I haven't been able to confirm the actual data values just yet, struggling with getting a long enough trace and also using the serial decoder on the scope simultaneously. To really do it I may have to use my Saleae Logic...

[omegahacker]

1st - keep the default timings in place... do not tamper with the internal clock. This won't help at all. My only suggestion is to lower the DMX frequency in the interface's configuration. (in-output tab, universe with the open interface, press the config button. Lower the DMX frequency to 30Hz (or 24, or somewhere in between).

I reset the mastertimer to the default 50, then raised the USB DMX output rate to 50Hz (there's no indication there's a settings window buried in there...), but also already had lowered the output channel count to 256 since we're only using 200. I verified the DMX output on my Rigol MSO4024, that it was running at 30Hz (actually 33ms so 30.303...Hz), and now it's running at 50Hz.

2nd—If you are in a time-critical situation, you must upgrade your DMX interface. With many fade times, your interface will struggle to keep pace. Your CPU is doing all the critical DMX calculations. If you have a 'pro' interface, this CPU load will be dissolved. Stuttering movements prove this. Every step in the fade equals a new strain of DMX calculations. As long as they are not confirmed, the QLC+ engine waits a bit - in your case, it has inconsistent timing.

This does not make sense to me at all. The "pro" DMX interface does not take on any of the DMX calculations from QLC, so there should be no difference at all there. Once QLC calculates the entire batch of output channels for a universe, it hands it off an array of up to 512 bytes to the output driver, which is then responsible for exporting it to interface hardware. In the case of the FTDI driver for the Open DMX, the output function is strictly asynchronous (as evidenced by the use of the D2XX driver) and possibly(?) threaded, which means that once that array of bytes gets to the driver, the main QLC engine should be effectively completely out of the loop. This is effectively confirmed by my scoping of the DMX output, where the timings are pretty much rock solid at 33ms per frame, and even with a very consistent break and MAB waveform. None of these operations should have any effect on the QLC main timing.

The only difference a "pro" interface would have is that there would be less USB interaction between the host and the interface - for the FTDI chip to generate break and MAB there have to be several USB packets back and forth, and then it sends a single packet with all of the DMX values (I did a USB packet capture yesterday confirming this, I can do it again and post a screenshot if desired). With a "pro" interface this traffic would be reduced to "just" the DMX universe data, and the interface would generate the break/MAB internally.

(background: I have implemented DMX over FTDI from scratch in the past, I have written both DMX input and output code for microcontrollers, written an entire USB stack for a microcontroller, and done a wide range of other unrelated things with various FTDI chips)

3rd—Do not use the show component. QLC+ is not a timecode machine. Your timing with the show will surely be off with more than the desired offsets.

I looked at the Show Manager last night and am still confused about how it's supposed to be used, but AFAIK it definitely is not going to help me building this setup anyway.

I've dealt with these kinds of timing issues before in various circumstances, and what it almost always boils down to is the use of relative delays without regard for the processing time between the end of the previous delay and the beginning of the next, rather than recalculating an "absolute" time for each delay relative to the initial start time.

(background: I invented the GStreamer multimedia framework and have done quite a bit with audio and video codecs that are extremely time-sensitive)

I have to wonder though - if QLC is not supposed to be time-accurate, why does it allow a user to specify millisecond-level precision for fade/hold/duration? That very clearly sets the expectation that I should be able to rely on it to actually implement millisecond-level accuracy for things....

4th—If timing is critical, build in some manual 'go' steps inside the chaser. This will introduce an additional workload for the operator.
But I would start by lending a pro interface. Ask your local distributor if you may use it as a demo.

I don't think we'll have any choice but to use that tactic, so hopefully we can build in enough manual cues to avoid significant time drift without adding so many that it bogs down the operator too much...

I borrowed a Chauvet DMX-AN2 adapter and set it up in ArtNet mode, and I got the same problem - 1:40 chaser ran in 1:45. I can try sACN mode but I don't expect anything different.

[Yestalgia]

I have to wonder though - if QLC is not supposed to be time-accurate, why does it allow a user to specify millisecond-level precision for fade/hold/duration? That very clearly sets the expectation that I should be able to rely on it to actually implement millisecond-level accuracy for things....

I want to be very clear here: The millisecond-level accuracy is fantastic and very accurate. What GGGS is talking about is the "Shows" function of QLC+ which does leave a little to be desired if using Windows and matching lightshows with long videos for example.

[GGGss]

I'm following this thread with absufabulous interest, because of the number of mishaps tracing back to the 'open' interfaces.
@Erik: With the open interfaces, the CPU is solely responsible for the correct DMX timing at the output, right? If the CPU gets other burdens to deal with for one reason or another, the DMX train timing may be inaccurate. A MAB taking too long, will break the whole DMX packet? And are we referring to the v4 of DMX Artistic Licensing protocol? In a Github comment, there was a mention of QLC+ not being adapted to v4...