C# Debug/Compile Question

[CycloWizard]

I'm used to MATLAB where execution is line-by-line in the order of my code. However, I'm not sure if all languages are this way and I was wondering specifically if C# would somehow run things out of order. I ask because I have a program receiving data from a piece of hardware on interrupt. Once the data is received, it is thrown into a buffer. I added a "1" in the last column of the buffer at the very end of the interrupt function to check and see if it made it all the way through. When I look at the data, all of the rows have a "1" at the end, but some data seem to be missing and I'm not sure how that's possible unless the code is executing in an order different from the code.



edited for even greater confusion.

 

[degibson]

Originally posted by: CycloWizard
I'm used to MATLAB where execution is line-by-line in the order of my code. However, I'm not sure if all languages are this way and I was wondering specifically if C# would somehow run things out of order. I ask because I have a program receiving data from a piece of hardware on interrupt. Once the data is received, it is thrown into a buffer. I added a "1" in the last column of the buffer at the very end of the interrupt function to check and see if it made it all the way through. When I look at the data, all of the rows have a "1" at the end, but some data seem to be missing and I'm not sure how that's possible unless the code is executing in an order different from the code.



edited for even greater confusion.

C# probably won't reorder, but the underlying hardware will. Try inserting a memory fence between the last buffer write and the writing of the '1'.

 

[Markbnj]

I think I would look at other possibilities before assuming that the flag is actually being written to the buffer before the data. There may be some reordering at various levels (opcodes, assembler, CIL) based on optimizations, and if you have more than one thread running that obviously opens up another can of worms. But in general instructions in C#, as in most other languages, are executed serially according to the order of occurence of statements, and the order of precedence in expression evaluation. There's no mystery voodoo going on that will make the instruction that writes the 1 execute before the instruction that fills the buffer, if that is not the order in which they were originally written. It would be kind of hard to write programs if there were .

 

[CycloWizard]

Thanks guys. The rep from the company that makes the hardware said C# will cache instructions if it doesn't have time to perform them before being interrupted. Thus, the 1 was probably getting written after the fact and wasn't a good indicator of whether the instructions were finishing in time. I either need a way faster computer or a new approach, and since I don't have any money, a new approach it is.

 

[degibson]

How frequent is the interrupt?

 

[CycloWizard]

Data is sent every 2 ms, though the recorded time interval is very erratic. I have trimmed down what my program is doing elsewhere and it seems to be behaving much better now. I got rid of the real-time data display and just have it collecting data quietly in the background which isn't as pretty, but I can always look at the data afterwards. I was just confused before because it seemed to be getting through, but wasn't.

 

[Markbnj]

Originally posted by: CycloWizard
Thanks guys. The rep from the company that makes the hardware said C# will cache instructions if it doesn't have time to perform them before being interrupted. Thus, the 1 was probably getting written after the fact and wasn't a good indicator of whether the instructions were finishing in time. I either need a way faster computer or a new approach, and since I don't have any money, a new approach it is.

That's an interesting way to put it. I would have said that whatever was happening when the interrupt occured is suspended, the state of the stack is saved, execution jumps to the interrupt handler, and then after it finishes the state of the stack is restored. Now, if the code in the interrupt handler can't be completed before another interrupt occurs that can cause any number of issues. In the 80x86 world we always issued CLI on entry to an interrupt handler and then either IRET, POPF, or STI right before exiting, to prevent the reentrance into the interrupt handler code. I'm not sure what the right approach is in modern assembler, but I would echo degibson's question about the interrupt frequency.

 

[Markbnj]

I definitely wouldn't update the display from within the interrupt handler. Should be relatively easy to check for changes and update it outside the handler. But then I have never dealt with interrupts using C#, so I could be full of crap.

 

[CycloWizard]

I had it updating the display based on a separate timer. It was getting data every 2 ms, but it would only update the form every 250 ms. This may have conflicted with the file I/O, which appeared to be running on my second core. I haven't explicitly controlled the threading, but C# seems to be handling it pretty efficiently on its own, which is nice.

 

[degibson]

2ms isn't much time for a user-level interrupt handler. I think Markbnj was entirely correct about the display, however -- especially if those 'realtime updates' are synchronous. I could easily envision just one synchronous OS call take >2ms.

 

[CycloWizard]

The data is sent from the controller via UDP, which I believe is asynchronous. The code retrieving the data from the packets is only a few lines and doesn't appear to have any problems executing in 2 ms anymore (now that I connected directly to the hardware rather than going through a hub connected to the internet). It looks like the form updates and receiving other packets that were not from the hardware were the primary culprits.

If you guys know of a better way to handle this sort of thing, which I'm sure you probably do, I'm all ears. I am pretty much making this up as I go along.

 

[Markbnj]

500 samples per second... it doesn't sound ridiculously high speed, but then I don't do much (or any) of this kind of thing any more. Back in the day I did because if you didn't then you couldn't use most hardware in your programs . Probably there are data acquisition applications that have to run a lot faster than that. So I'd guess if you limit the code to the bare minimum needed to service the interrupt and update the related data structures you'll be fine. In a case like this I would be looking at an architecture with two threads: one running in the background and servicing the interrupt; and a GUI thread that I can run (possibly at a lower priority) to gather up info for display and collect input from the user.

 

[degibson]

Originally posted by: CycloWizard
The data is sent from the controller via UDP...

AHA! Now we're getting to the bottom of things. You should be aware of the following, if you're not already.

1) There's a helluva lot of overhead, from the OS, between the packet coming in off the wire to the time your handler executes. Maybe ~ms. So you probably have less than 2ms to service your packet.

2) If you're using native POSIX signals (probably not w/ C#, but JIC), you should make sure to mask and unmask signals at entry/exit if you're using that route. That'll prevent reentrance and force the OS to buffer the packets.

3) If you're using some other API, find out if its reentrant or not. Reentrance would look a lot like some mysterious thread traipsing all over your memory.

4) You may want to verify the inter-interrupt time. It'll probably have a lot of variance -- even if the controller is sending every 2ms.

Also, I entirely agree with Markbnj's thread suggestion. Use a slave thread to run the rest of the application. Just make sure you appropriately sleep your threads instead of spinning.

 

[CycloWizard]

Thanks again guys. I have pretty much given the entire system to taking in the data and periodically writing it to file. Once the experiment is finished (i.e. no more incoming data), the data is displayed. I started to think about how similar commercial instruments generally do it, and this is the approach they use. I figure I'm not smarter than them, so I'll not try to reinvent the wheel here, especially with a low budget and an inexperienced programmer like me running the show.

 

[imported_Dhaval00]

I am curious (been some time since my Network Analysis course) as to why you're using UDP - I mean I understand that there are no acknowledgments going back and forth like TCP, so the noise is reduced, but UDP can drop packets and overwhelm the network. So in your case, Cyclo, how are you guaranteeing data delivery?

 

[CycloWizard]

Good question. The hardware I'm getting the data from is running some feedback loops at very high speeds, so I don't want to interrupt it while it's doing what it needs to do. It can upload UDP without using any of its CPU cycles, allowing me to get the data without losing control of the process. Since that's what it gives me, that's what I'll take. I could use TCP, but at the cost of execution speed. The feedback loop can run at around 5 kHz if I leave it to its own devices, so that's what I'm shooting for.

Bandwidth utilization is very low and the hardware is connected directly to the computer, so the network isn't really stressed at all. I think each packet is only around 140 bytes or so such that, even at 500 packets/second, there isn't a whole lot of transfer going on.

 

[degibson]

Streaming = UDP.
(UDP FTW!)

On uncontended ethernet, packet loss rate for that bandwidth will be really low.

 

[EagleKeeper]

W/ respect to the RealTime display.

The human can only process events down to the 20-25HZ level. There is no need to try to update data display at a faster rate than twice that amount.

Many RTD in aircraft try to update at 30HZ

 

[CycloWizard]

Originally posted by: Common Courtesy
W/ respect to the RealTime display.

The human can only process events down to the 20-25HZ level. There is no need to try to update data display at a faster rate than twice that amount.

Many RTD in aircraft try to update at 30HZ

Right, which was why my "real time" display was only really running between 4-10 Hz. It was running on a separate timer that triggered every 100 or 250 ms.