Logomatic (LPC2138) SD card writing speeds

I have bought the Logomatic and made a little change in source code. The goal of this operation was to increase writting speed to SD card, but It seems that Logomatic cannot write faster.

I have a similar device built on Atmel AT128MEGA running on 16MHz with FAT library already written. The device is capable to sample and log 8 channels at max. speed of 1.5ksample/s. The author used an external SRAM for FIFO, the explanation was that writing 512bytes on SD card takes 1-2ms, while reallocating the FAT table makes an delay of 250ms, this occurs every 500kB on a card of 128MB, therefore a FIFO buffer of at least 4kB could solve this problem.

I have implemented a circular buffer in Logomatic, but it overruns in less than one second. I used 8kB and then I tried 16kB, but the result was the same. I discovered that writing 512bytes on Logomatic takes more than 40ms as described in manual. Im little disapointed, since Logomatic uses LPC2138 running on 58Mhz with SPI speed approx 7.5Mhz with SD card of new generation with capability of 20MB/s and it is not possible to log with speeds already achived with an old SD card and slower MCU.

I would like to ask if there is somebody that tried to log data with LPC21xx with some different library (fatlib.c) or some other filesystem and what are the reached speeds, because I think that problem is in fatlib.c not in the LPC.

I have measured the frequency of SCK0 pin with a osciloscope and I have found that is near 500kHz. I don’t understand anymore, it should be 7.5Mhz.

Check your PLL settings.

Most important: writing speeds are dramaticaly low with some “old” 128 or 256Mbytes cards. On a 16M card I get a poor 20kbytes/s, then 150k/s only by changing the card to 1G, without any other mods.

Why? the difference is not in the data transmit time. At the end of sector write, the software polls the ready bit. This is very long on old cards.

Angelo

I have found the true initialization if SPI0 port. Its in HALayer.c , not in the main.c Initialize() routine:

S0SPCCR = 0x08; // set up SPI clk to be pclk/8, this is the fastest we can go

S0SPCR = 0x30; //master, msb first, first clk edge, active high, no ints

In main() routine at the begining calls fat_initialize(), then fat_initialize() in fatlib.c calls HALayerInit(). In HALayerInit() SPI0 port is set again:

void HALayerInit(void)

{

IODIR0 |= CS;

PINSEL0 |= 0x1500;

//S0SPCCR = 32; // SCK = 1 MHz, counter > 8 and even

//S0SPCCR = 60;

S0SPCCR = 120;//slowed down for new SD cards (Pete, 9/11/07)

// CHPOL = 1, CPHA = 1

S0SPCR = 0x38; // master, msb first, first clk edge, active high, no ints

}

So 59Mhz/120 =491kHz true speed of SCK. S0SPCCR should be an odd number and never less than 8. I have tried 8,10,12,14,16. At divisor of 16 SCK speed is 3.6Mhz and works, on other higher speeds card hangs on. The signal on scope was not a clean quadratic pulse, maybe it needs a buffer or simply some pull-ups.

But now I have another problem: It seems to me that I try to write to a file when the previous record is not finished yet. This ready bit that you are talking about would solve the problem, but I dont know how to get it.

Coyote,

Would you please direct me to the AT128MEGA based logger device you are talking about ? Sound’s very interesting.

Thank you.

This device was developed on Faculty of electrical engineering in Ljubljana, its called MED-X. This is the only thing that can I say, but it works fine as I have described: eight channels 12bit could be sampled at a maximal frequency of 1.7Khz before it hangs. Logomatic could do the same, I attempted to add a FIFO buffer, because the this is the main solution and attach on ssp port digital accelerometer or ADIS16350. Until now it doesnt like to log data with normal speed.

I looked on SD card specs and found that SD card should be initialized with 400kHz, once it is initialized you can boost the speed. So the HALayer.c remains as it is, after execution of fat_initialize in main.c i have add a line:

S0SPCCR = 0x08;

Now the card (I got the Toshiba SD) works fine even at high speed (7.3Mhz), but it works fine just before writing 212-th sector (108kB) then it slows down and buffer overruns. The same occurs if there is another file on a card so i suspect that some calculation in fatlib is wrong. I will try also some different card.

O.K. I made it! Now my LOGOMATIC can log six channels with a speed of 1.6kHz.

The problem is in Fatlib.c in function fat_write. It does not compute correctly switching of clusters. Since I put in a buffer exactly 512 bytes I have patched fat_write in a way that it only writes single sectors not multisectors and it works. I think I will correct the error in fatlib.c soon and it will work as it should also with multisectors.

Coyote973:
O.K. I made it! Now my LOGOMATIC can log six channels with a speed of 1.6kHz.

Excellent! I’ve had a “high speed version of logomatic” on my to-do list for several months. I’m glad you beat me to it :slight_smile:

Would you be willing to post your code (either whole or a ‘diff’)?

Glen Overby

How can I post it to you?

(Note: My first attempt to post this ran afoul of some sort of filter. The message claimed I used a forbidden word but it might have just been spam filters.)

I have a high speed version of the code in work right now. The first step will be to replace the general purpose FAT code it uses now with something a bit more specific. Thrashing of the FAT table just slows things down so it will only be updated at file close time.

Second up is to shift to using the multiple sector write command.

If that isn’t fast enough the last step is to pre-erase most of the card prior to starting the write.

As I don’t have a Log-O-Matic in hand (yet) I have just been looking over the code. But I found some of the code in main.c offended me and I just couldn’t leave it alone. Along the way I did a number of things:

  1. Replaced the ten copies of the ADC code with one general purpose subroutine.

  2. Replaced the many copies of code to add a byte to the output buffers with a single subroutine that inserts a byte into the buffers.

  3. Replaced the three copies of code that checks for a full buffer to write with a single subroutine.

  4. Replaced the buffer code with something that can handle N buffers.

  5. Removed the calls to fat_flush(). These appear to generate a write to update the directory and were called after every write.

  6. Deleted a lot of variable declarations that were never used.

  7. Decreased power consumption by entering idle mode when waiting.

  8. Stomped a nasty bug in the code to read the configuration file. For some reason it declared a 256 byte buffer and then read 512 bytes into it. Never a good thing to do.

As I have never worked with an ARM processor before there is a good chance that I have missed some little detail plus the usual chances of bugs. But at this point the code at least compiles. I don’t see any way to post it here. It can be found at:

http://home.earthlink.net/~schultdw/logOmatic

I have started on improving the firmware. My first working version just cleans up main.c and adds a 16K buffer along with boosting the SPI clock speed. The result is an improvement in data rate from about 1400 SPS total to about 10,000 SPS total. Or 10 channels enabled at 1,000 SPS each. This is in binary mode of course.

I tested this with a couple of versions of micro-SD cards and a Sandisk Ultra II. The Ultra II wasn’t any faster.

I have posted my version of main.c and a .hex file at:

http://home.earthlink.net/~schultdw/logOmatic

I have my code screaming along at 90,000 SPS now. The code still needs cleaning up and something has to be done about the blinking lights consuming so much power whenever the program stops. I will post it once that is done.

Of course this matters not so much as the V1 Log-O-Matic is now obsolete. Just after I bought one of course.

thanks for your code and i am still looking forward for your new 90KSPS code :slight_smile:

I dont think logomatic V1 is obsolete just yet, its cheaper and like me other people may be inclned to stick with it for a while. logomatic adds usb but i dont need that.

i am an electroncie engineer so when it comes to coding i am not really good at it. i was hoping you could help me with the prob.

I have the rawley compiler installed. and i can BUILD the code into hex that is given by sparkfun. buy my hex files dont run ( well the leds dont blink) while the original sparkfun hex files work. I dont know what i am doing wrong. there are no erros during build

can you help with this?

thanks

TK

teekay_tk:
thanks for your code and i am still looking forward for your new 90KSPS code :slight_smile:

Actually I am almost finished with a version that will do much more than 90KSPS. I just need to iron out a FAT bug that showed up with large (>100MB) files.

I dont think logomatic V1 is obsolete just yet, its cheaper and like me other people may be inclned to stick with it for a while. logomatic adds usb but i dont need that.

The V1 is no longer being made so that pretty much makes it obsolete for new work.

i am an electroncie engineer so when it comes to coding i am not really good at it. i was hoping you could help me with the prob.

I am an EE as well, but I have been messing around with computers ever since I built an ELF in high school. Heck, one of my EE courses had us programming a KIM-1.

I have the rawley compiler installed. and i can BUILD the code into hex that is given by sparkfun. buy my hex files dont run ( well the leds dont blink) while the original sparkfun hex files work. I dont know what i am doing wrong. there are no erros during build

can you help with this?

thanks

TK

I had this problem as well and it had me stumped for a while. Buried in the target specific startup source file and in a message posted here, is a little detail. You need to tell the target specific startup code that you want to run your program. Its default action is to drop into an infinite loop.

Really!

In the Project Explorer window select the “Philips_LPC210X_Startup.s” file. (Read the comments in the source for the explanation of this decision.) Then look in the Properties window below that. Scroll down to “Preprocessor Options”. Add “STARTUP_FROM_RESET” to the preprocessor definitions. This tells the startup code to run your code.

thanks

that solved the problem, would have taken me ages to get there

will look forward to your new code, will it also have a new fat file with it?

I found the last (probably not) bug and it only cost $40 to fix it. I refuse to say how much time I wasted on trying to find the problem in my code. :slight_smile:

The problem was not with my code but with my ancient card reader. The smoking gun was finding directory and FAT data in the middle of a 100MB data file. Plus it changed when I wrote new files to the card using the computer. I was thinking that I really needed a new USB 2.0 reader to deal with the big files anyway.

It probably isn’t quite ready for prime time but it works so here are links to the source tree and a quick description of the new operating mode I have added.

http://home.earthlink.net/~schultdw/log … mode3.html

http://home.earthlink.net/~schultdw/log … 4_0.tar.gz

160,000 SPS looks to be about the upper end unless I work on the ADC IRQ routine some more.

Comments, suggestions, bug reports, etc. are requested.

O.K. I took a look on your code and I have to say that is much different than the original, so I don’t understand much, but I would like to ask you some questions:

  1. Have you implemented a buffer ( like cache) that temporarily stores data during the write process? (I mentioned in earlier posts that this is essential)

  2. I have tried in mode 2 with 10 channels and sampling freq 10kHz, the device did not answer any error ( buffer overrun), but it surely cannot write such speeds.

  3. How can I add in the MODE2ISR() routine a simple counter which increases every interrupt and writes on the card, instead of ADC channels. In this way it could be possible to test the maximum writing speed.

Coyote973:
O.K. I took a look on your code and I have to say that is much different than the original, so I don’t understand much, but I would like to ask you some questions:

  1. Have you implemented a buffer ( like cache) that temporarily stores data during the write process? (I mentioned in earlier posts that this is essential)

  2. I have tried in mode 2 with 10 channels and sampling freq 10kHz, the device did not answer any error ( buffer overrun), but it surely cannot write such speeds.

  3. How can I add in the MODE2ISR() routine a simple counter which increases every interrupt and writes on the card, instead of ADC channels. In this way it could be possible to test the maximum writing speed.

  1. See files buffer.c and buffer.h The #defines in buffer.h set the buffer size and number. Don’t get so carried away that you don’t leave space in the 32K of RAM for things like stacks and the other data.

  2. I have tested mode 2 to 90,000 SPS (10 channels at 9,000Hz) with no trouble. If there is no space in the buffer, new data is dropped so for testing in mode 2 I used a simple program to scan for the ‘$’ marker in the data file. Grounding the inputs helps a lot for this:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

main(int argc, char **argv)
{
  int i, c, lastchar;

  int val;
  int length;
  int mode = 0;

  length = 10;
 
  if(argc == 2)
    {
      fprintf(stderr, "%s", argv[1]);
      length = atoi(argv[1]);
      if(length < 1 || length > 10)
        {
          fprintf(stderr, "Data length parameter out of range: %d\n", length);
          exit(1);
        }
    }

  lastchar = 0;
  i = 0;
  length = length * 2 + 2;
  while( (c = getchar()) != EOF)
    {
      i++;
      if(mode == 0)
        {
          if(c == '
  1. The mode 2 code is replaced with the simple program you are looking for if you define TESTSD. It also checks for buffer over run and marks the next buffer. This is scanned by:
#include <stdio.h>

main()
{
  long offset;
  int i, c, count;

  count = offset = 0;

  while( (c = getchar()) != EOF)
    {
      offset++;

      if(count == 4)
        {
          if(offset % 512)
            continue;
          else
            count = 0;
        } 
      switch(c)
        {
        case 0xde:
          count = 1;
          break;
        case 0xad:
          if(count == 1)
            count++;
          else
            count = 0;
          break;
        case 0xbe:
          if(count == 2)
            count++;
          else
            count = 0;
          break;
        case 0xef:
          if(count == 3)
            {
              count++;
              printf("%04x\n", offset/512);
            }
          else
            count = 0;
          break;
        default:
          count = 0;

        }
    }
}

I measured about 400KB/sec write speed. This decreases when running the ADC because not all of the CPU cycles are available to the data storage code. && lastchar == ’


3) The mode 2 code is replaced with the simple program you are looking for if you define TESTSD. It also checks for buffer over run and marks the next buffer. This is scanned by:

§DISCOURSE_HOISTED_CODE_1§


I measured about 400KB/sec write speed. This decreases when running the ADC because not all of the CPU cycles are available to the data storage code.)
            {
              mode = 1;
              i = 0;
            }
        }
      else 
        {
          if(i == length)
            if(c == '
  1. The mode 2 code is replaced with the simple program you are looking for if you define TESTSD. It also checks for buffer over run and marks the next buffer. This is scanned by:
§_DISCOURSE_HOISTED_CODE_1_§

I measured about 400KB/sec write speed. This decreases when running the ADC because not all of the CPU cycles are available to the data storage code. && lastchar == ’


3) The mode 2 code is replaced with the simple program you are looking for if you define TESTSD. It also checks for buffer over run and marks the next buffer. This is scanned by:

§DISCOURSE_HOISTED_CODE_1§


I measured about 400KB/sec write speed. This decreases when running the ADC because not all of the CPU cycles are available to the data storage code.)
              {
                i = 0;
                putchar('\n');
              }
            else
              mode = 0;
          else if((i&1) == 0)
            {
              val = ((lastchar << 8) | (c & 0xff))&0x3ff;
              printf("%d ",val);
            }
        }
      lastchar = c;
    }
}
  1. The mode 2 code is replaced with the simple program you are looking for if you define TESTSD. It also checks for buffer over run and marks the next buffer. This is scanned by:
§_DISCOURSE_HOISTED_CODE_1_§

I measured about 400KB/sec write speed. This decreases when running the ADC because not all of the CPU cycles are available to the data storage code.

I have added 8 counters in mode2isr() for test and it showed that it overruns without showing any error, but it overruns for sure. I have tested at 10kHz down to 3kHz and at last ended 2.6kHz, but now I got some other problem: it wont write more than 16kB and showing :

SD_Init()= 0

fat_init() returned 2

Configuration complete.

Timer freq = 2600

Mode = 2, processing data

Stopping

Error during multi block write. num = 8, 0xffffffff

The error appears after removing the card, after few seconds like it is waiting something in endless loop instead of writing the last record.

Coyote973:
I have added 8 counters in mode2isr() for test and it

showed that it overruns without showing any error, but it overruns for

sure. I have tested at 10kHz down to 3kHz and at last ended 2.6kHz, but

now I got some other problem: it wont write more than 16kB and showing :

SD_Init()= 0

fat_init() returned 2

Configuration complete.

Timer freq = 2600

Mode = 2, processing data

Stopping

Error during multi block write. num = 8, 0xffffffff

The error appears after removing the card, after few seconds like it is

waiting something in endless loop instead of writing the last

record.

I really don’t know what the problem could be although it might be specific to the memory card you are using. I only have four different types of cards (64MB Sandisk MMC, 128MB Kingston micro-SD, 2GB Patriot micro-SD, and 2GB Sandisk Ultra II SD) so I could not perform exhaustive testing.

The only thing I can think of is that it is hanging in a busy state as those are the obvious infinite loops. You could add a timeout to the busywait() routine but the problem is that there aren’t any good recovery schemes. If an error occurs, the program is dead.

If you can’t nail down the problem and fix it, just drop the buffer size down to 1*SECTOR_SIZE and the multi-block write code will not be used.