Jump to content

As of July 17, 2015, the LabJack forums here at forums.labjack.com are shut down. New registrations, topics, and replies are disabled. All forums are in a read-only state for archive purposes.

Please visit our current forums at labjack.com/forums to view and make new posts. To post on the current forums, use your labjack.com login account. Your old LabJack forums login credentials have been retired. There are no longer separate logins for labjack.com and LabJack forums.


Python Streaming and Real Time Logging and Computation

  • Please log in to reply
10 replies to this topic

#1 missionviraj

  • Members
  • 6 posts

Posted 07 July 2014 - 03:07 PM



I just got a LabJack U6 and am trying to write a program using LabJack Python. The basis for my script/program is the streaming example given in the examples folder of the source code. I've got the Mux80 and four of the CB37 boards. I'm going to be using over 30 T-type thermocouples to measure temperature data. I basically want to log the data into a CSV or excel and then graph these against time. The area where the logging will happen I imagine is located in place of the two print statements.

 if r['missed'] != 0:
                missed += r['missed']
                print "+++ Missed ", r['missed']

            # Comment out these prints and do something with r
            print "Average of" , len(r['AIN0']), "AIN0," , len(r['AIN1']) , "AIN1 reading(s):", 
            print sum(r['AIN0'])/len(r['AIN0']) , "," , sum(r['AIN1'])/len(r['AIN1'])

            dataCount += 1
            packetCount += r['numPackets']

Since there is so much data and it may take time to convert the voltages to temperature readings, log and then graph in real time, I wanted to know the best way to add in these steps without missing samples/packets. Would I have to start some sort of parallel process or thread and then in place of those two print statements, I just add the 'r' dict to some global queue that the thread keeps working on at its own pace? Thanks for any help or suggestions. 



#2 LabJack Support

LabJack Support
  • Admin
  • 8677 posts

Posted 07 July 2014 - 03:15 PM

Before going into further detail, are you sure you want to stream?  Stream is usually only used for perhaps 100 scans/second and faster, which seems very fast for temperature signals.


Also, if you are acquiring raw thermocouple signals, you should be using the +/-100mV range, which means you are limited to the slower speeds you see in Table 3.2-3:



#3 missionviraj

  • Members
  • 6 posts

Posted 07 July 2014 - 04:20 PM

Thanks for the super quick response. I was following the section 3.0 Operation which states, "Stream mode is generally used at 10 scans/second or faster." The only reason I went to stream mode is because I thought that it was more accurate in terms of getting values at set times since the internal clock or scan frequency is within 30 ppm. 


I'll probably scan at 10 scans/second  or 15-20 at most so I suppose I can go to command and response mode. So, if the 100 number as you mentioned is correct, then I can just use the getAIN function? How would I set the scan rate or at least get it somewhat close to a consistent scan or sampling rate? And then would the (pseudo) time stamp be 0 + 1/(scan frequency) * number of iteration? In either case I'd still need to something with threading to do the V to T conversion and graphing, right?


If based on this information (10-20 scans/second effectively of ~25-30 thermocouples), you recommend stream mode, what would you do? In either case? Thanks for the help!



#4 LabJack Support

LabJack Support
  • Admin
  • 8677 posts

Posted 08 July 2014 - 01:50 PM

I would recommend command/response for slower scan rates like you want. You can use a timer to perform your readings every 100 ms (10 scans/second) and in the timer code get the timestamp from the system (your computer). Depending on how long V to T conversions and graphing takes in your code, for example thermocouple readings, conversions and graph time > 100 ms, you will want to do them in a different thread.


I wouldn't use getAIN as that only performs 1 AIN reading per USB command/response. For better efficiency when reading more than one AIN, use getFeedback with multiple AIN24 IO commands. Note that the Feedback command can only fit 14 AIN24 commands in one command/response, so you will need to split your 30 AIN readings for the thermocouples into 3 getFeedback calls. For an example on reading multiple analog inputs with getFeedback, including converting the binary readings to a voltage, take a look at the u6allio.py example. AIN24 class/IOType documentation can be found in the u6.py source code or using Python help ("help(u6.AIN24)").


For command/response speeds with different configurations take a look at the tables here. Speeds are for 1 Feedback command/response:




For a range of +-100 mV and 100 scans/second with 30 samples, try a a resolution index of 1 first and increase the resolution index if there is too much noise. Increases in resolution index will slow down reading times.

#5 missionviraj

  • Members
  • 6 posts

Posted 08 July 2014 - 05:22 PM

Hi thanks for the response. Is there a reason you recommend command/response over streaming? The one main advantage that I can think of is that I won't have to worry about figuring out how to parse the dictionary and different size lists from each packet for logging and graphing. 


Going with the command/response I have this right now: 

#Define acquisition parameters
resolutionIndex = 2
gainIndex = 0
settlingFactor = 0
differential = False
channelList = range(30)

feedbackArguments = [0]*len(channelList)
latestAinValues = [0]*len(channelList)

#Create Feedback Argument List
for i in range(len(channelList)):
	feedbackArguments[i] = ( u6.AIN24(channelList[i], resolutionIndex, gainIndex, settlingFactor, differential) )

#Break up into chunks of 10 channels per feedback command sent
chunks = [feedbackArguments[i:i+10] for i in range(0, len(feedbackArguments), 10)]

#Gather data and return list of readings in order of channel list
result = []
for i in chunks:

for i in range(len(channelList)):
	latestAinValues[i] = d.binaryToCalibratedAnalogVoltage(gainIndex, result[i], resolutionIndex = resolutionIndex)

#Put in queue for other thread to process. 
print latestAinValues
print len(latestAinValues)

The above returns latestAinValues which is a list of voltages recieved in the order of channelList (which is just reading channel #0-30 in this case just for now).


So I need to execute over the portion right before 'result = []' to the end at a specific rate of say 10 scans/second. From asking around, I'd want to put this into a queue and then have another thread chomp on it for converting, logging, graphing, etc. I am unfamiliar with Timer but it only executes once. How would you set it up so that a single thread loops at a specified rate? Then when I set an event, it will stop. I've seen an example of a thread created every time but this is less than ideal as I want a single thread to communicate and put stuff in a queue for another thread to work on. Any suggestions or ideas? Thank you so much. 

#6 missionviraj

  • Members
  • 6 posts

Posted 09 July 2014 - 11:02 AM

Also, I don't know how accurate Timer is but I'd like to avoid creating a new thread for ever feedback command. If this is the only way, I'd love to get your advice on how to proceed. 


Is Timer based on my computer's clock? 


Would something like the following work?

from time import time, sleep

while True:
    startTime = time()
    yourFunction() #Sample 30 thermocouples and put in Queue for other thread for conversion, logging, graphing
    endTime = time()-startTime

Thanks again for all the help.

#7 LabJack Support

LabJack Support
  • Admin
  • 8677 posts

Posted 09 July 2014 - 01:24 PM

We recommend command/response over streaming for low scan rates because command/response is typically less complicated and easier to design your program around. For command/response when you want your data you just perform a call to get it instead of streaming where you wait for the U6 to sample the data, buffer it and then you read from the U6 stream buffer. Also, streaming is limited to 25 channels, so in your case you couldn't use it for 30 thermocouples.


As for your last post, your loop and timing look good and is an alternative to using Timer. I suggested Timer since in many languages you can configure it run every X milliseconds easily, but your loop code will do that as well and takes into account the thermocouple reading times. The only suggestion I have is to make sure your sleep time is not a negative number to prevent unexpected delays:


sleep(max(0, 1/ScanFrequency-endTime))


Looking through some of the Timer source code, its timing is based on time.time() and delays on time.sleep() which use the system's time.

#8 missionviraj

  • Members
  • 6 posts

Posted 11 July 2014 - 05:40 PM

Thanks for the help so far. I've got a simple working implementation creating three sets of feedback commands each with 10 thermocouples and getting the binaries and converting to voltages. This works great and I loop ever 100 milliseconds (10 Hz scan rate) based on the loop format shown in my previous post. 


So the next issue is we've now added an additional component to our test setup and we have a flow meter that is an open collector circuit. It outputs a pulsed frequency response and we'll want to measure the frequency. From what I've tried to gather, it seems like from this post (https://forums.labja...?showtopic=1105) that we'll likely want to go with a counter or F to V circuit or use a counter. Unfortunately the options using hardware timed in stream mode are out of the option since I'm using command/response mode. 


We'll likely encounter lower frequencies in the range of 5Hz to 30Hz but the flow rate itself will be changing much, much slower. So in LabJack python, it seems like I'll have to do something to the effect of d.configIO(EnableCounter0 = True) and then create a Counter0 feedback command and submit that in d.getFeedback to get the count. The resultant frequency would be Counts / (Sample period of Counter0). Is this correct? 



Time (ms) Resolution (Hz)
100 10
1000 1
10000 0.1

You can see that if you are trying to measure a frequency from 0-10 Hz, with 0.1 Hz resolution, you must wait 10 seconds between counter reads. If however you are reading a frequencyfrom 0-1 MHz, and require 10 Hz resolution, you can get updated readings every 100 milliseconds.


Can you explain the above please? Can I just acquire the counts at the same rate as I am acquiring the thermocouples (10Hz)? But than that seems to provide 10Hz resolution (according to the table above) which in our range of 5-30Hz would be unacceptable.

#9 missionviraj

  • Members
  • 6 posts

Posted 11 July 2014 - 06:40 PM

Sorry, no need to explain how the resolution vs time table works. Basically 1 count will be 1/(time between sampling) so that is your resolution. However all my other questions remain. Thanks.

#10 LabJack Support

LabJack Support
  • Admin
  • 8677 posts

Posted 14 July 2014 - 12:48 PM

You frequency calculation using counters looks correct, but to be more specific it would look like:


frequency = (current count - previous count) / (current time - previous time)


However at slower frequencies and a 10 Hz read rate, the counter solution will not be ideal. I suggest using a timer configured for 32-bit period measurement (mode 2 for rising edges and mode 3 for falling edges). Here you get the number of U6 hardware clock cycles between the current and previous rising/falling edge with configurable clock frequency. Take a look here for timer hardware documentation:





The PWM-looping demonstrates timer configuration (configure for timer mode 2 or 3 instead), and the commented out feedback command (the example uses Modbus but provides low-level function equivalent calls in the comments) used to update the timer value can also be used to read the timer value ("d.getFeedback(u6.Timer()"):



#11 LabJack Support

LabJack Support
  • Admin
  • 8677 posts

Posted 14 July 2014 - 12:51 PM

Also, if you have access to a Windows computer you can use LJControlPanel to test timer configurations before coding.

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users