xbee series 1 to Arduino Mega byte loss

Hi,

I am sending potentiometer data from the transmitting xbee to the receiving xbee just fine. This is verified by the spark fun explorer with the receiving xbee pluged directly into my computer. I am consistently getting 10 bytes of good data all the time. However when I connect the receiving xbee to the Arudino Mega running the following program. Then do a serial print so I can see the data on the arduino side of things, I have 10 bytes of good data, but after a few seconds I loose the last two bytes which are the most important as they include the payload. The pattern is that I loose the last two bytes after a few seconds of Serial.print. The first few seconds the data is correct, then say 10 seconds later last two bytes are not there. I know the complete data is being transmitted from the sending xbee to the receiving xbee but somewhere I am loosing those 2 bytes in getting it into the arduino with serial.available and serial.read.

Can you please help? Ultimately when I get it working I need the x and y data to calculate motor speed and direction…

Thank you.

Transmitting xbee:

Xbee series1

One pot

3.3 volts power supply

Receiving xbee:

Xbee series 1

Arduino Mega r3

Logic level converter between xbee and arduino

The arduino code:

int data= 0;

void setup()
{
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop()
{
 if(Serial1.available() > 0)  // checks for data
 
  {
   data = Serial1.read();  // reads data from Serial1
    
    if(data <= 0x0F)    // prints a leading zero on Serial.print if
                        // Hex is less than or equal to 15 as arduino 
                        // wont print a leading 0.  This took some time to figure out with help.
      {
        Serial.print("0");
      }
    
    Serial.print(data,HEX);  // this is where I loose a couple of bytes
                             // of data.  The last two or 3 bytes
    Serial.println("");
  } 
}

I allready told you in another thread. You have two serial lines operating at 9600 baud. On the input you retreive 1 byte from the Xbee. From that you make 4 bytes: sending 2 bytes to represent the hex value in ascii characters, and follow up with a Serial.println(“”) which is two hidden bytes for the carriage return and linefeed. You are sending more than you receive. Now this is no problem if the XBees transfer only a few bytes ever so often. But if they send a continuous stream near capacity, then somewhere in the line, a buffer overflows and bytes are getting lost.What are your byte rates along the path?

Slow down the Xbees by making the transmitting-Xbee measure the pot just once a second. Does the byte loss happen also?

HI Valen,

Thank you for responding. I did try your suggestion before, and changed the arduino program to Serial.write instead of Serial.print without a println, but when looking at the ardino terminal screen, I could not read the data so I could not compare what the xbee is sending vs what the arduino is getting. Hence I changed it back to serial.print so I could read the hex values.

  1. Is there a way to read HEX values (which the xbee is sending) on the Arduino terminal screen using serial.write? I need to parse the HEX bytes and throw away all but the x and y values to use for motor speed. I guess I need to verify this is happening on the arduino terminal screen. I am at work now, but tonight I will try and slow the xbee transmitter to 1 time per second.

  2. Can you help me understand what happens in the buffer? Each byte moves up in the buffer after each serial.read() command? Then when there is space at the end of the buffer, the new info sent from the xbee moves into the back end of the buffer. If there is not enough space, the rest of the sent byte is lost as there is no room in the buffer? If so, then the xbee send must be slower than the arduino serial.read? otherwise the buffer will overflow?. Or there must be enough room in the buffer to accept a full byte of sent data every time or part of the byte will be lost?

Thank you so much.

Scott

I understand (from surfing the net) that the serial monitor in the Arduino IDE is quite limited in it’s use. It only shows the bytes as ascii character. I never used any Arduino myself, so don’t consider me as an expert on it. But there should be plenty of terminal programs on the web that can be used to see what is received on a serial port, and show it as decimal, hex, ascii or even binary. Personally I like this program: https://sites.google.com/site/terminalbpp/ Or since you are using Xbees anyway, use the XCTU Terminal tab.

In stead of opening the Arduino Serial monitor, I suppose you can open the com-port on the terminal program. Except, you would be entering any characters you send to the Mega/Xbee in the terminal program.

How exactly Arduino uses the buffers is something I don’t know, but in general serial buffers are made to be in a ring-type fashion. This Wiki explains it much clearer than I can.

http://en.wikipedia.org/wiki/Circular_buffer

While your program sends into the other serial port the 4 characters that are needed to represent 1 received byte, the receiving serial port connected to the Xbee keeps getting new bytes. These end up in the 64 byte buffer until the program has a chance to execute the next serial.read command. Then one is taken from it, but leaves the other 3 for the next loop. But again, this 1 byte that is taken from the receive buffer takes the time of 4 to send to the pc. And again the input buffer grows with another 3 that it can’t process in time. Eventually the buffer reaches the max level. Whether the Arduino Stream class simply overwrites the old data, or generates some kind of runtime-error I don’t know. I guess the former, since microcontrollers don’t really have the user-interfaces to report run-time errors to the user. And the Arduino site http://arduino.cc/en/Reference/Stream doesn’t say anything about generating errors events when buffers overrun.

With 9600 baud you ultimately have 960 bytes per second that you can send. (10 bits: assuming 1 start, 8 data, no parity, 1 stop bit) You send 4 characters per receive byte. So your program can process 240 bytes from the Xbee per second maximum. Now, count up the number of bytes in a API packet. Divide 240 by the size of a packet (frame start all the way to Checksum byte) and you have the maximum number of packets you can handle in the stream to your serial monitor. But set a lower sample rate, since there is always some latency involved.

Or you set the baudrate of the serial connection to the pc to 38400 bauds and the bottle neck is solved. Make sure downstream is quicker than upstream.

HI Valen,

Thank you for responding and your help. It worked. I increased the IR to 0x64 or 10HZ or 100 milliseconds, and now I can read all 10 bytes not including the 7E, packet length and checksum. This is great. Now I can parse the data and only use the payload.

I tried an experiment with timing the byte calculation on the arduino. See below. It takes about 3.1 milliseconds per byte read. yet the IR is .1 seconds or 100 milliseconds on the xbee. It is interesting that I loose bytes when I add this new byte read section. yet when I take it out, i get all the data sent from the xbee and can read it on the arduino serial terminal. Even with 100 milliseconds on the IR for the xbee, I can loose data by just adding a few extra lines in the arduino code. This baffles me…

I will try a new terminal tomorrow. Got home late from work tonight. Thank you again. I am sure I will be back soon with another quesiton… I sure enjoy this new stuff…

int data= 0;
int oldMicro = 0;
int time = 0;

void setup()
{
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop()
{
 if(Serial1.available() > 0)  // checks for data
 
  {
    oldMicro = micros();
   data = Serial1.read();  // reads data from Serial1
    
    if(data <= 0x0F)    // prints a leading zero on Serial.print if
                        // Hex is less than or equal to 15 as arduino 
                        // wont print a leading 0.  This took some time to figure out with help.
      {
        Serial.print("0");
      }
    
    Serial.println(data,HEX);  // this is where I loose a couple of bytes
                             // of data.  The last two or 3 bytes
   Serial.print("\t");                         
   time = micros() - oldMicro;   
   Serial.println(time);    
    
  } 
}

Micros() returns an unsigned long, but you put it into an int. Are you sure you are not loosing significant bits in the calculation?

I’m curious what your time log looks like.

HI Valen,

I changed the code to use unsigned long. I could not copy the Serial monitor as right clicking the mouse gives no options at all, for some reason, so I will type some data here. I assume that 3108 = 3.108 milliseconds.

Data Time

7E 3108

00 3104

0A 3108

83 3108

12 3108

34 3108

1C 3108

00 3108

01 3112

02 3108

02 3108

7E 3108

Notice that all the 10 bytes are not there.

unsigned long  data= 0;
unsigned long oldMicro = 0;
unsigned long time = 0;

void setup()
{
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop()
{
 if(Serial1.available() > 0)  // checks for data
 
  {
    oldMicro = micros();
   data = Serial1.read();  // reads data from Serial1
    
    if(data <= 0x0F)    // prints a leading zero on Serial.print if
                        // Hex is less than or equal to 15 as arduino 
                        // wont print a leading 0.  This took some time to figure out with help.
      {
        Serial.print("0");
      }
    
    Serial.print(data,HEX);  // this is where I loose a couple of bytes
                             // of data.  The last two or 3 bytes
   Serial.print("\t");                         
   time = micros() - oldMicro;   
   Serial.println(time);    
    
  } 
}

Now I changed the code and hid the timer

unsigned long  data= 0;
unsigned long oldMicro = 0;
unsigned long time = 0;

void setup()
{
  Serial.begin(9600);
  Serial1.begin(9600);
}

void loop()
{
 if(Serial1.available() > 0)  // checks for data
 
  {
    oldMicro = micros();
   data = Serial1.read();  // reads data from Serial1
    
    if(data <= 0x0F)    // prints a leading zero on Serial.print if
                        // Hex is less than or equal to 15 as arduino 
                        // wont print a leading 0.  This took some time to figure out with help.
      {
        Serial.print("0");
      }
    
    Serial.println(data,HEX);  // this is where I loose a couple of bytes
                             // of data.  The last two or 3 bytes
   //Serial.print("\t");                         
  // time = micros() - oldMicro;   
   //Serial.println(time);    
    
  } 
}

Now I have all 10 data bits with this change, but can’t see the timer. Just that little bit of code removed

will allow the arduino to read all the bytes in the buffer. Weird.

7E

00

0A

83

12

34

1C

00

01

02

00

02 payload reading the pot, ADC in: ox 02 0D = 637 637/1023 = .623 .623(3.3V) = 2.05 volts Yes!!!

7D payload

98

7E

Yes, adding the loop duration to the serial monitor increased your data output. And caused dataloss on the input side, the bytes in the Xbee packet.