Reading binary data

Hi, Im a very new user to the arduino, and i need help getting started with a project.

I have a sensor that outputs 77 bits of binary code in 140ms,

I believe it is manchester coding, this data contains 6* 2 digit hex numbers. I need to be able to send this binary data to the arduino . initially it would be good to show the incomming data on the screen.

1111111001010011010101000110011111001101000110110110111111011001110000000

11111110 01 01001101 01 01000110 01 11110011 01 00011011 01 10111111 01 10011100 00000(spaces added by me to show data split)

This converted to hex should give 4D,46,F3,1B,BF,9C

I believe the 1111111 start sequence is posibly to inform the receiver that data is being sent and the 00000 at the end informs the receiver that is the end of the string.

The end result im looking for is the arduino to read the binary data and convert it to hex and display either on screen or lcd display.

The sensor has two output connections that i have connected to earth and Digital pin 2.

I now strugling to make anything happen. Im not sure of the Baud rate. I have tried some code obtained from various sources but nothing seems to happen.

Like all tasks in life it seems the first steps are the hardest.

Any help greatly received.

I would start with learning how to sample an Interrupt-pin (I think pin 2 is one of them) , and log the changes in duration using the micros function as a timebase. As the pulse lengths(pulse high) and space lengths (signal low) may be of different duration. Depending on the baud rate of your serial connection to the pc, it might not be quick enough to send the pulse lengths in real time. So you may have to store it in memory first. Or only make a histogram count out of it. Once you know how long a bit takes, you can rebuild your program to interpret the signal changes as bits on the fly. Don’t worry about converting it into hex yet. Just put them into the right sequence into bytes and sent them out to your pc for further analysis. Once you are sure where the header ends, how the hex values are seperated, and how many bits are left for the footer of the bitstream, should you worry about slicing the stream up into individual hex codes.

Also it helps if you allready know what the datastream contains.

These can be interesting challenges, but if you want help thinking about it, you’ll need to provide more information.

For example, what sort of a sensor is it? That narrows down some of the possibilities.

Why do you think the data are Manchester encoded? Pulse width modulation is more common.

Six two digit hex numbers can be encoded in 48 bits, so what do you think the other bits do?

There are lots of web sites and forum posts (including Spark Fun) on decoding the bit streams from commercial RF weather sensors, IR remote controllers, etc. so spend some time with Mr. Google.

Sparky99:
I have a sensor that outputs 77 bits of binary code in 140ms,

How often does your sensor spit out these 77 bits ? IOW how much 'dead time' is there btw the 140ms periods ?

FWIW I think Valen’s approach should work well. The interrupt could be set to happen on both edges (rising & falling), yielding pulse times. Given the bit timing, the number of high and low bits in each high and low timing period could be determined. Thus the bitstream can be reconstructed … and then parsed and decoded into a hex representation.

Are you quite sure that this is not inverted standard serial data @ 600 baud with 8 bits, 1 start, 1 stop, no parity?

If so, an inverter and a standard UART input will do the job.

Thanks for all the help. At present I use a logic anaylyser to study the output signal from the sensor, I then plot the output trace and manually read off the binary data, from this data i then convert the binary code to hex. Allthough vey time consuming its 100% accurate, I have included the latest trace available if that would help to identify what is being sent, I have converted the data to hex digits.11111111100101101101010010010001000100110100111001010101011101001111000 (data deciphered from trace)

1111111110 01 01101101 01 00100100 01 00010011 01 00111001 01 01010111 01 00111100 0 (spaces added by me to clarify)

Converted to required hex vaulues.

01101101 = 6D

00100100 = 24

00010011 = 13

00111001 = 39

01010111 = 57

00111100 = 3C

The sensor only outputs this data when asked for, selected by user pressing a button, no other data is sent.

It wasn’t quite apparent what the signal was without the actual signal.

Is this thing an IR transmitter or supposed to be one?

You really could give us a hint.

So, it is a variant of standard serial, with inverted data, 8-N-1-1 running at 500 baud.

The zero is a 1 msec pulse with 1 msec rest.

The one is a 2 msec rest.

(It’s not Manchester, not enough transitions.)

You just need to trigger an interrupt on the falling edge.

You don’t care about the rising edge because it will always happen 1 mS later.

Hi sorry to not answer, The sensor is fitted to a telescope in an observatory. All the motions, pan, tilt, lens settings are measured very accurately and the data is fed into a processor. The processor outputs the data as shown. The telescope is of German design with the electronics made by a Korean company, both companies now finished trading. When the observer focuses on something of interest he operates the “trigger” and the processor outputs the data as shown.

These 6 hex numbers are the exact reference to enable the telescope to be aimed and focused on the same point again.

I tend to agree with Renate but this is not really standard serial, inverted or not. I agree that a “1” bit is a high followed by a low (one bit period) and a “0” bit is high for two half-periods. The way you have spaced out the bits, the pattern is not completely consistent with 8-N-1-1, which would be 10 bits/character, beginning with a 1 bit, followed by 8 data bits and ending with a zero bit. The exception is an extra 0 bit after the preamble.

The bit timing is not clear to me. What is the scale of the horizontal axis in the plot you posted? Note that the “highs” in the plots are always multiples of one “high” timing while the lows are always single (not consistent with Manchester encoding).

To keep things simple, you don’t need interrupts to decode this signal. With an Arduino, you can use input polling and micros() to keep accurate track of time. Some variation of the following procedure will work: wait for a falling edge on the input and start the clock. Assuming a bit time of 2000 microseconds wait perhaps 500 microseconds so that you can sample at a convenient position and then sample every 2000 microseconds thereafter, recording a 1 bit if the input is low and 0 bit if the input is high. Use the logical OR and bit shift operators (<<1 or >>1 as appropriate) to accumulate the data bytes. For this method to work, the bit period has to be known quite accurately (to a few microseconds) as any timing error is cumulative and the sampling will eventually get out of sync with the message.

If you are really just beginning with Arduino, study the “blink without delay” tutorial to see how to use micros() to keep track of time. http://arduino.cc/en/Tutorial/BlinkWithoutDelay

Finally, keep in mind that “hex” is just one representation of a number. The bytes could be equally well represented by decimal integers or ascii characters.

Let us know how it goes!

Well, it is enough like standard serial that you could probably pass it through a pulse stretcher and into a standard UART.

The values of the RC should be about right, but will depend on the exact switching points of the Schmidt trigger.

Check with a scope, there should be no glitches in the signal, nor stretched too long.

You can adjust the R.

The “preamble” would just show up as a break.

The pause after the “preamble” is just a pause, no big deal.

The rest of the stuff is just 8-N-1-1.

The data out of the UART would have to be complemented.

The baud rate seems to be 500, but we could use more exact measurements.

Whether you actually decide to implement it this way or not, it would be an interesting test.

Sparky99:
I have included the latest trace available if that would help to identify what is being sent, I have converted the data to hex digits.11111111100101101101010010010001000100110100111001010101011101001111000 (data deciphered from trace)

1111111110 01 01101101 01 00100100 01 00010011 01 00111001 01 01010111 01 00111100 0 (spaces added by me to clarify)

Converted to required hex vaulues.

01101101 = 6D

00100100 = 24

00010011 = 13

00111001 = 39

01010111 = 57

00111100 = 3C

r

The sensor only outputs this data when asked for, selected by user pressing a button, no other data is sent.

Do the above hex values represent what the receiver decoded or what you decoded from the bitstream ? IOW how sure are you that those values are 100% correct ? I ask because I don't count 77 clock periods in the attached waveform pic.

(click on to open)

I looked at the pictured bitstream as NRZ-I and come up with a different set of 1’s and 0’s. Looking at my annotated pic below and dividing the stream into bytes (and adding 2 0’s to the end) I get;

11111111 11001011 01101010 01001000 10001001 10100111 00101010 10111010 01111000

0xFF 0xCB 0x6A 0x48 0x49 0xA7 0x2A 0xBA 0x78

(click on to open)

http://en.wikipedia.org/wiki/Non-return … o_Inverted

Non return to zero, inverted (NRZI) is a method of mapping a binary signal to a physical signal for transmission over some transmission media. The two level NRZI signal has a transition at a clock boundary if the bit being transmitted is a logical 1, and does not have a transition if the bit being transmitted is a logical 0.

Sorry for the dalay in replying been working away.

Firstly i must apologise i think i have confused those that have answered. Im not looking to understand what the trace is i allready know that, i have been deciphering these traces for over 4 years using a logic analyser. I can get 100% accuracy, listed below the information is correct

I have included the latest trace available if that would help to identify what is being sent, I have converted the data to hex digits.11111111100101101101010010010001000100110100111001010101011101001111000 (data deciphered from trace)

1111111110 01 01101101 01 00100100 01 00010011 01 00111001 01 01010111 01 00111100 0 (spaces added by me to clarify)

Converted to required hex vaulues.

01101101 = 6D

00100100 = 24

00010011 = 13

00111001 = 39

01010111 = 57

00111100 = 3C

This data string is transmitted in 140ms.

What i need help on is getting the arduino to decipher the data steam and produce the hex characters, on the pc or on a seperate display.

I have played around with baud rates an various settings but can get nothing to give any kind of out put. The sensor output is connected to Ground and Digital pin 2.

many thanks for your help.

This

Sparky99:
What i need help on is getting the arduino to decipher the data steam and produce the hex characters, on the pc or on a seperate display.

I have played around with baud rates an various settings but can get nothing to give any kind of out put. The sensor output is connected to Ground and Digital pin 2.

many thanks for your help.

This

Ok, but let's understand what the Arduino 'sees'. When you say '1111111110' to start the bitstream, the trace, which is what the Arduino must translate, is really like the pic below.

Now I’ve assumed that the pic included the entire waveform. Even so I get an extra ‘1’ compared to your translation. What level is the line sitting at prior to the 1’st bit, a high level or a low level (not seen in trace above) ? Can we assume these levels are 5v and 0v ?

So what’s your method ? That’s part 1 of programming the Arduino.

Lets further clarify what you have done and what are trying to do:

1- You can read in the serial data steam into bytes. yes?

2- You now want to display (send to PC) the value of these bytes.

3- You want to see these bytes in a Text format such as 6C.

4- To do that you need to convert each nibble (4 bits) of each byte into an ASCII character representing the hex value.

5- This is a very common thing to do so search for “hex to ASCII conversion” for arduino.

OR is this not the problem?

1- Is it that you can not get the arduino to read the serial data stream correctly?

2- For that search for “bit banged” serial receiver code or IR remote serial receiver code.

3- This would be an arduino code project.

waltr:
Lets further clarify what you have done and what are trying to do:

1- You can read in the serial data steam into bytes. yes?

To be clear my understanding is that he has a logic analyzer to capture the bitstream. He 'manually' translates (his wording) the waveform into the 71 bits (not 77, a typo ?) he lists. I assume he does this by looking at the LA output, not unlike what I did in my mark-up off his trace. That's certainly a PITA and error prone. So I assume that's step #1, and the trickiest part. Replacing the LA and human effort w/an Arduino, certainly a do-able thing.

Parsing the bits into the bytes desired is then trivial.

Lets see if the OP concurs.

Trying to answer your questions I cannot get the data to be recognised by the aurdino I have played with various settings Baud rates but nothing happens.

Re questions above.

  1. With a logig analyser i can read the serial data (dont know what into bytes refers)

2 & 3) the end result is to get the pc to display the uncoaded data as 6 pairs of hex numbers

  1. not sure

  2. converting hex to ascii is not the problem

Question: Is it that you can not get the arduino to read the serial data stream correctly?

I cannot get the arduino to read it at all.

Levels are 5v and OV

There may be an extra 1 in the preamble, i dont start manually deciphering until i get to the first 01

I have played with various settings Baud rates but nothing happens.

I cannot get the arduino to read it at all.

Still have questions on these two statements.

Baud rate? Are you trying to use the aurdino’s UART? or Softserial? These will not work since the serial data stream is not asynchronous serial. You MUST use a custom Bit-bang serial receiving code like how code for an IR remote receiver works.

With the serial data you have Baud Rate is a meaningless term. A proper term would be either ‘clock rate’ or ‘sampling rate’.

So, you need to write code that samples the input pin at a known period of either a multiple of the data rate or at the data rate. Since you have a logic analyzer have the code output what it samples. Then connect the analyzer to both the input and output. These should be the same, with a slight delay of the output. Then the aurdino’s code is properly sample the input data. Next is having the code turn the input sampling into ‘1s’ and ‘0s’ and shifting these into bytes (8bit chunks) and storing into a buffer. After that code pulls the data from the buffer, converts it to ASCII and sends it to the PC.

Perhaps this example will give you some ideas about how to proceed. The following is part of some C code I wrote for a PIC processor to decode the bitstream from a weather station remote sensor.

The Arduino code can look similar.

Basically the program waits for a pulse edge and then times the next pulse transitions. First it looks for a series of “0” pulses followed by a “1” (the preamble), then decodes the message. A long pulse is a binary one and a short pulse is a binary zero. In your case, the bit times are constant and you would be looking for an absence of a transition during the bit period to distinguish a 0 from a 1 (just as you do by hand). The code below uses a hardware timer, but on the Arduino you have the function micros() that does the same thing.

// the following define expected interval timing and allowed 
// uncertainty for timer tick rate 64 uSec

#define deltaT 5
#define InRange(TVAL, TSETUP, dT) ((TVAL<=(TSETUP+dT))&&(TVAL>=(TSETUP-dT)))

#define TrfON0 31               // 2 ms RF ON time, zero bit 
#define TrfON1 94				// 6 ms RF ON time, one bit

BitValue = 0;
while(1)

   {  //look for message preamble
      
	  while ((PINB & RF_IN) == RF_ON); //RF is on, wait for RF off

      Ton = TCNT0;     // Read ON timer value
	  TCNT0 = 0;     // reset timer

	  while ((PINB & RF_IN) == RF_OFF); //time the off period 
      
      Toff = TCNT0;  
      TCNT0 = 0;

      if (InRange(Ton, TrfON0, deltaT) && InRange(Toff, TrfON1, deltaT)) PreambleCnt++;
	  if (InRange(Ton, TrfON1, deltaT) && InRange(Toff, TrfON0, deltaT)) BitValue=1;
	  if ((PreambleCnt > 6) && (BitValue == 1)) break;

// invalid RF on pulse length?

	  if (!InRange(Ton, TrfON0, deltaT) && !InRange(Ton, TrfON1, deltaT))
	  { PreambleCnt=0; BitValue=0; }

    } //end while (1)


    PORTB |= LED;  //LED on, preamble detected

// if OK so far, get rest of message

     BitCnt = 1;
	 SI_msg.l = 0; //ignore start bit, clear message buffer

/*
** loop through next 28 bits and build message
** Decide one/zero by checking the RF_ON length (2 or 6 ms)
*/
     while (BitCnt < 29)
     {  
         while ((PINB & RF_IN) == RF_ON); //wait for RF_OFF
		 
		 TimerValue = TCNT0;
		 TCNT0 = 0;
		 
		 BitValue = 2;  //assume invalid

		 if(InRange(TimerValue, TrfON0, deltaT)) BitValue=0;
		 if(InRange(TimerValue, TrfON1, deltaT)) BitValue=1;
 
 		 if(BitValue == 2) ErrCode |= 1; //invalid RF ON pulse width    
 
//shift in bit

		SI_msg.l <<= 1;
		if (BitValue == 1) SI_msg.l |= BitValue;

//wait for rising edge, with timeout upon valid end of message (RF_OFF >> 6 ms)

         while (((PINB & RF_IN) == RF_OFF) && ((TCNT0 & 0x80) != 0x80)); //0x80 = 8.2 ms

		 TimerValue = TCNT0;  // and check that RF off time was valid
		 TCNT0 = 0;

		 if (BitCnt < 28)  //unless it is the last bit, flag bad RF_OFF period
			{
			 if ((BitValue == 1) && !(InRange(TimerValue, TrfON0, deltaT))) ErrCode |= 2;
			 if ((BitValue == 0) && !(InRange(TimerValue, TrfON1, deltaT))) ErrCode |= 4;
			}

         BitCnt++; 

     } //end while BitCnt

Again let me ask if the level stays at 5v after the end of a transmission and until the start of the next transmission ? That is the start of the bitstream is a falling edge, from 5v to 0v.

Alternately you should be able to use the pulsein() function to measure an array of ‘high’ pulse times. The first 8-10 of these can be used to set the bit width. This can then be used to count the number of 0’s. Then, post-transmission, the bitstream couldvbe reconstructed.

http://arduino.cc/en/Reference/pulseIn