Home
JAQForum Ver 24.01
Log In or Join  
Active Topics
Local Time 15:30 26 Nov 2024 Privacy Policy
Jump to

Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.

Forum Index : Microcontroller and PC projects : GPS data logger issues

Author Message
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:56pm 21 Nov 2012
Copy link to clipboard 
Print this post

Hello

I have been working on a GPS logger for some months.

I was having good success with LCD and LED displays for data.

I even did some basic TRIG.

I would like to share some observations in the hope someone may be able to help.

I have had problems with

1. Data corruption on the serial line
2. SD card writes after a long period of time.

With the data corruption on the serial line, this shows up as extra charcaters in the NMEA data stream.

I have tried processing character by character and making up a message based on commas and * which identifies the end of a NMEA line.

I have also just recently tried the LINE INPUT command as well.

Here is a sample of what is going on

$GPRMC,234336.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*70
$GPRMC,234337.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*71
$GPRMC,234P338.000,A,2720.2571,S,15302.5943,E,0.00,183.56,20 1112,,,A*7E
$GPRMC,234339.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*7F
$GPRMC,234340.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*71

See the extra "P" snuck in there somehow.

I tried playing with the serial buffer - did not really help.

Is it too much for the DuinoMite to read NMEA, print to VGA and save to SD ?

I have after about 12 hours, an error and the system stops writing to the SD card.

I have tried opening the file, and leaving it open and just append - I lost a whole lot of data after a power failure - the file was not closed off.

I also tried each NMEA line, writting to the SD card. open append and close.

Am what I am doing a big ask ? I did everything in assembly language besides write to an SD CARD then I went looking and found DuniMite.

The NMEA is from a MOD-GPS at 19200. I have a buffer of serial of 1024.

I am keen to get this stable as it is great solution.

Its just these two odd behaviours I can't explain

The power supply is fine.

I am printing to VGA the NMEA as well

Here is the code, its pretty simple and displays odd charcaters and bombs out on SD card writes now and then


file$ = "GPSLOG.LOG"
Open "COM3:19200,1024" As #1
Do
Line Input #1,msg$
Print msg$
Open file$ For append As #3
Print #3, MSG$
Close #3
Loop


EXT is used - DuinoMite Mini

- Andrew VK4TEC -


Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 12:04am 22 Nov 2012
Copy link to clipboard 
Print this post

Here is the other way I do it

Get character by character from the serial port

//



file$ = "GPSLOG.LOG" ' initial filename
max =20
Dim arg$(max)

Open "COM3:19200,1024" As #1

Open file$ For append As #3
Do

nmea_sentence

'Print msg$
x$ = Input$(1, #1)
msg$ = msg$ + x$
x$ = Input$(1, #1)
msg$ = msg$ + x$

If arg$(0) = "GPRMC" Then
Print msg$
file$ = arg$(9) + ".LOG"
GoTo savefile
EndIf

If arg$(0) = "GPGGA" Then
Print msg$
GoTo savefile
EndIf

Rem save to SD card

GoTo andrew

savefile:

'file$ = arg$(9) + ".LOG"
'Open file$ For append As #3
Print #3, MSG$
'Close #3



andrew:

Loop

Rem -- SUBS --


Sub nmea_sentence
Do
msg$ ="$" ' subroutine start
Do While Input$(1, #1) <> "$" : Loop ' wait for the start
For i = 0 To max
arg$(i) = "" ' clear ready for data
Do ' loops until a specific exit
x$ = Input$(1, #1)
' Print x$
msg$ = msg$ + x$ ' get the character
If x$ = "," Then Exit ' new data field, increment i
If x$ = "*" Then Return ' we have all the data so return
arg$(i) = arg$(i) + x$
'Print arg$(i) ' add to the data
Loop
'Print arg$(i) ' loop back for the next char
Next i ' move to the next data field
' Print "Corrupt data..." ' exceeded max data items
'Print msg$
Loop

End Sub


Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 01:44am 22 Nov 2012
Copy link to clipboard 
Print this post

I have tried something

1. Reduce the GPS rate to 9600
2. Turn off all but $GPRMC and $GPGGA
3. Gone back to the character by character mode


- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
cwilt
Senior Member

Joined: 20/03/2012
Location: United States
Posts: 147
Posted: 05:56am 22 Nov 2012
Copy link to clipboard 
Print this post

Andrew,
Have you tried a different GPS? I had a Duinomite Mega and a GPS shield from Sparkfun using similar software design running for months without issue.

This is the shield.

Does the error happen at completely random times?
 
Talbit
Senior Member

Joined: 07/06/2011
Location: Australia
Posts: 210
Posted: 11:47am 22 Nov 2012
Copy link to clipboard 
Print this post

Andrew,
See my article on my GPS clock somewhere under my name. I'm using a Garmin GPS 16HVS running at 19200, 1Hz spitting out $GPRMC only. I haven't spotted any problems. Running at that speed have you got good layout with short interconnecting wires? I'm using the original Maximite with around v3.1 I think.
TalbitEdited by Talbit 2012-11-23
Talbit
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 04:05pm 22 Nov 2012
Copy link to clipboard 
Print this post

Got some GPS I can try

GPS18LVC
GPS18 5Hz
GPS72

Speed changes did not make a dif, maybe it is GPS

- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 12:24am 23 Nov 2012
Copy link to clipboard 
Print this post

Change of pace

Dumped the MOD-GPS

Hooked up GARMIN GPS185Hz

Locked it down to 19200

Locked it down to $GPRMC

So far so good

Maybe GPS was cactus ?

- Andrew -

Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:16am 24 Nov 2012
Copy link to clipboard 
Print this post

I have now worked out my GPS issues.

I now need to attend to the SD CARD issue.

After a long time. The program will stop

at open file$ for append as #3
Error: Cannot write to file.

Does that mean the file is not closed off properley ?

Is it too big ?

Is it that I need some pause statements ?

The SD card is written to many times. and quickly.

- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:29am 24 Nov 2012
Copy link to clipboard 
Print this post

Is there such a thing as software commanded reboot ?

A software watchdog timer ?

- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
bigmik

Guru

Joined: 20/06/2011
Location: Australia
Posts: 2914
Posted: 12:45pm 24 Nov 2012
Copy link to clipboard 
Print this post

  vk4tec said   Is there such a thing as software commanded reboot ?

A software watchdog timer ?

- Andrew -


You could always try connecting an unsed ouptut pin (pulled high via a 10k or 4k7 OHM resistor) to the RESET.. when you want to reset the MM direct the pin to go LOW and bingo it kicks itself up the ..... errr ... well it resets itself.

The MM should reconfigure the pin as an INPUT and the pull up would keep it high until the MM has control again.

Regards,

Mick

Mick's uMite Stuff can be found >>> HERE (Kindly hosted by Dontronics) <<<
 
James_From_Canb

Senior Member

Joined: 19/06/2011
Location: Australia
Posts: 265
Posted: 02:30am 25 Nov 2012
Copy link to clipboard 
Print this post

Are we talking about the code you posted above? That code only opens the file before the main DO statement, which is the first few fractions of a second of processing. You should never get that message a long time after you've started the program.

Would you please post your current code?

Thanks

James
My mind is aglow with whirling, transient nodes of thought careening through a cosmic vapor of invention.

Hedley Lamarr, Blazing Saddles (1974)
 
TinkersALot
Regular Member

Joined: 20/11/2012
Location: United States
Posts: 72
Posted: 03:01am 25 Nov 2012
Copy link to clipboard 
Print this post

How large is the file when the program quits?
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:08am 25 Nov 2012
Copy link to clipboard 
Print this post

I think I might have fixed it

Instead of one big file or daily files, I have gone to hourly files.

Has not bombed out yet.

- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:21am 25 Nov 2012
Copy link to clipboard 
Print this post

The files at the moment are about 2 MB

- Andew -
Andrew Rich VK4TEC
www.tech-software.net
 
bigmik

Guru

Joined: 20/06/2011
Location: Australia
Posts: 2914
Posted: 06:04pm 25 Nov 2012
Copy link to clipboard 
Print this post

  vk4tec said   The files at the moment are about 2 MB

- Andew -


Thats Interesting Andrew,

Maybe you have your SD card formatted at FAT12 which has a maximum file size of 32MB (which at 2MB per hr is about 16hrs use..)

If you formatted with FAT16 you can get 2GB files or FAT32 you can get 4GB files..

Regards,

Mick
Mick's uMite Stuff can be found >>> HERE (Kindly hosted by Dontronics) <<<
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:25pm 25 Nov 2012
Copy link to clipboard 
Print this post

What is odd, is when the program bombs out, I can restart it and it keeps going for a bit more.

Its not as if it reaches a limit and then stops all together

Maybe it is like the variable to write to the card or the offset for the SD card becomes too big.

It sometimes bombs out trying to open, as if it did not close properley.

Well at the moment it has not missed a beat at 2 MB hourly files.

- Andrew -
Andrew Rich VK4TEC
www.tech-software.net
 
vk4tec

Senior Member

Joined: 24/03/2012
Location: Australia
Posts: 239
Posted: 11:44pm 25 Nov 2012
Copy link to clipboard 
Print this post


Andrew Rich VK4TEC
www.tech-software.net
 
MicroBlocks

Guru

Joined: 12/05/2012
Location: Thailand
Posts: 2209
Posted: 01:36am 26 Nov 2012
Copy link to clipboard 
Print this post

  vk4tec said   What is odd, is when the program bombs out, I can restart it and it keeps going for a bit more.

Its not as if it reaches a limit and then stops all together

Maybe it is like the variable to write to the card or the offset for the SD card becomes too big.

It sometimes bombs out trying to open, as if it did not close properley.

Well at the moment it has not missed a beat at 2 MB hourly files.

- Andrew -

How about compressing/filtering the data. That would save a good amount of data.
An entry i save is like this:
Timestamp,longitude,latitude,direction,speed
some real data:
19f2wq,4.516478,51.812278,N,0

The timestamp is compressed by using an offset and base64 coding of the timestamp.

It is about 32 bytes on average.
If you have 1 per second an hour would be about 32 * 60 * 60 = 115200 bytes.
You can have about 17 hours worth of data in that 2 MB.
I found for vehicle tracking once a minute is sufficient while driving and every 15 minutes while parked.
It is really a datasaver. A vehicle transmitting that data over GPRS is on average 1.2MB for a month!
All application differ but they do have in common to find out what data is really needed, not what is available as that often is just too much and can slow down the processing of that data a lot.

Edited by TZAdvantage 2012-11-27
Microblocks. Build with logic.
 
Print this page


To reply to this topic, you need to log in.

© JAQ Software 2024