TheBackShed.com

Forum Index : Microcontroller and PC projects : SD card write issues

Page 1 of 2

Author

Message

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 10:38am 30 Nov 2012

Copy link to clipboard

Print this post

This is really one for Geoff

I have been writing to an SD card on the DuinoMite Mini for some time in a GPS logger. I had a crook GPS which was replaced and is now 100% on the data.

I keep having program stops on card writes.

I have tried reducing the size to hourly files which is down to about 2 MB.

Is it that I am writing to fast to the SD CARD ?

It is not full and can bomb out at any time - about a day later.

I re-run the program and write to the same file append and it keeps going.

It sometimes happens on close statments and sometimes on open statements.

Is there any debug ? Is there some delays needed ?

I am writing at 5 HZ to the SD card.

If I can nail this issue my GPS logger will be 100% working

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

bigmik

Guru

Joined: 20/06/2011
Location: Australia
Posts: 2914

Posted: 05:27pm 30 Nov 2012

Copy link to clipboard

Print this post

Andrew,

Have you tried another SD card? A class 10 even? not that I think that the class would be the issue here.

Why not try to create a NEW file every time and then every xx Hours go to a routine which combines them into a new (larger) file and deletes the small files.. it might be some bug in the append being open for an extended time. It might help to debug the actual problem. The idea being that no file is open for more than a second or two.

Regards,

Mick

Mick's uMite Stuff can be found >>> HERE (Kindly hosted by Dontronics) <<<

cwilt
Senior Member

Joined: 20/03/2012
Location: United States
Posts: 147

Posted: 05:47pm 30 Nov 2012

Copy link to clipboard

Print this post

Andrew,

Are you using any interrupts?

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 07:25pm 30 Nov 2012

Copy link to clipboard

Print this post

Nope no interupts

I append to the file 5 times a second.

do

open
print #
close

loop

Trying a new card.

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

BobD

Guru

Joined: 07/12/2011
Location: Australia
Posts: 935

Posted: 07:39pm 30 Nov 2012

Copy link to clipboard

Print this post

If you think it is an SD card problem have you checked the error code(s) when it fails? See MM.ERRNO on P17 of the V4.1 manual.

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 11:14pm 30 Nov 2012

Copy link to clipboard

Print this post

Print MM.ERRNO = "10"

10 = Error eading from card

to a "Close #3"

Andrew Rich VK4TEC
www.tech-software.net

BobD

Guru

Joined: 07/12/2011
Location: Australia
Posts: 935

Posted: 11:38pm 30 Nov 2012

Copy link to clipboard

Print this post

Try making your program more robust. When you get an error then retry the operation but limit the number of retries.

Hardware possibilities,
change to a different SD card,
change to faster SD card,
check the supply volts to the SD card.

I recall reading somewhere that the SD card needs a minimum voltage to work correctly. I can't recall what it is but I had to mount an extra capacitor on the supply volts to my SD card socket as it was bombing out sometimes when accessed. I could believe that some brands would be worse than others if this was the problem.

cosmic frog
Senior Member

Joined: 09/02/2012
Location: United Kingdom
Posts: 284

Posted: 02:34am 01 Dec 2012

Copy link to clipboard

Print this post

Have you had a look a this thread? - http://www.thebackshed.com/forum/forum_posts.asp?TID=5247&PN =2

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 09:38pm 01 Dec 2012

Copy link to clipboard

Print this post

Error 11

11 = Error writing to file

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 11:26pm 06 Dec 2012

Copy link to clipboard

Print this post

Ok I have gone from CLASS 2 to CLASS 10 SD card

See what happens now

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 10:20am 07 Dec 2012

Copy link to clipboard

Print this post

Nope - stopped again

But I can see that the card is writing much faster

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 12:07pm 07 Dec 2012

Copy link to clipboard

Print this post

http://www.youtube.com/watch?v=WPemNZBihlc&feature=youtu.be

Video on project Edited by vk4tec 2012-12-08

Andrew Rich VK4TEC
www.tech-software.net

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 01:30am 09 Dec 2012

Copy link to clipboard

Print this post

Geoff gave me an idea

I had played with

1. Open and close the file every NMEA line
2. Open the file and then close the file on button press

Now I am trying

1. Open the file and close and re-open every minute

That way the most I ever loose is one minute

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

James_From_Canb

Senior Member

Joined: 19/06/2011
Location: Australia
Posts: 265

Posted: 01:17pm 09 Dec 2012

Copy link to clipboard

Print this post

Did you try BobD's suggestion of a limited number of retries of the open or close operation if it fails the first time?

If that didn't work, here's a work-around that might help for intermittent write failures:

If the open or close fails, retry a couple of times and if it still fails, store the data in memory.

Collect new data for the next minute.

Try writing the previous minute's data again. Then write the new minute's data.

That way, if there's some sort of timing/heat/voltage issue you've given the system a minute to get over it.

Another thought: If the file write continued to fail and the data is held in memory in an array you might be able to write it to a file from the command prompt after the program finishes.

James

My mind is aglow with whirling, transient nodes of thought careening through a cosmic vapor of invention.

Hedley Lamarr, Blazing Saddles (1974)

Geoffg

Guru

Joined: 06/06/2011
Location: Australia
Posts: 3194

Posted: 05:39pm 09 Dec 2012

Copy link to clipboard

Print this post

Andrew was opening and closing the file every time a GPS record was received.

The issue is that every time you close a file you force a write to the sector on the SD card that holds the file allocation table (FAT). If you open and close the file for each GPS message then at 5Hz GPS update rate you will force 1.8 million writes to this one sector in 10 hours.

Most SD cards have an endurance of only 100,000 writes to any one sector. So it is no good trying to recover, you need to reduce the number of open/closes. The best would be to simply open the file once and write to it as normal.

Geoff

Geoff Graham - http://geoffg.net

greybeard
Senior Member

Joined: 04/01/2010
Location: Australia
Posts: 161

Posted: 06:27pm 09 Dec 2012

Copy link to clipboard

Print this post

As an aside, at 5Hz that's 200mSec between samples. How far is the target device likely to have moved in 200mSec?
I could perhaps understand a model rocket moving along a bit every 200mSec but unless it's going into outer earth orbit it is only moving for a minute or so.

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 07:46pm 09 Dec 2012

Copy link to clipboard

Print this post

Interpolation the more points the smoother the curve

So now I think I have it stable I will run it for 3 days

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 09:56am 10 Dec 2012

Copy link to clipboard

Print this post

I want to wait until I have a date from the GPS to start file writes

Does anyone know how I would do that in code ?

Is there is there a

"if file4$ = "" then " loop ?

I need to think about when the date rolls over what would happen

I think i need to check if the date has changed, close the old file and start a new file with the new date as the filename.

Otherwise the program might be stuck trying to open and close an old file

Maybe I can check the file names and then close the old one and start the new one

Would only happen when the date rolls over

At the moment I have just one big file called "GPSLOG.LOG"

I would like daily logs "121212.LOG"

Maybe I can simulate in MM DOS ?

- Andrew - Edited by vk4tec 2012-12-11

Andrew Rich VK4TEC
www.tech-software.net

James_From_Canb

Senior Member

Joined: 19/06/2011
Location: Australia
Posts: 265

Posted: 11:34am 10 Dec 2012

Copy link to clipboard

Print this post

To check whether a file exists, use the FILES(fspec$) command.

To know whether the date from the GPS has changed:
When you start the program, store the current date from the GPS in a variable. Preferably in the same format as it is returned from the GPS, so comparison is easy.
Each time you read from the GPS check whether the current date is equal to the stored date.
When the date from the GPS changes, close the old file and start a new one.

It should be easy to simulate in MM DOS, just watch out for the CR LF issue :)

James

My mind is aglow with whirling, transient nodes of thought careening through a cosmic vapor of invention.

Hedley Lamarr, Blazing Saddles (1974)

vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239

Posted: 12:54am 12 Dec 2012

Copy link to clipboard

Print this post

I am having good success

I left the CCT running for three days now

Each minute the file is closed and opened again.

I need to start looking now at daily filenames.

Seems the bandwidth of writing to the file 5 Hz was upsetting things.

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net

Page 1 of 2

Print this page