Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me.
|
Forum Index : Microcontroller and PC projects : GPS data logger issues
Author | Message | ||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Hello I have been working on a GPS logger for some months. I was having good success with LCD and LED displays for data. I even did some basic TRIG. I would like to share some observations in the hope someone may be able to help. I have had problems with 1. Data corruption on the serial line 2. SD card writes after a long period of time. With the data corruption on the serial line, this shows up as extra charcaters in the NMEA data stream. I have tried processing character by character and making up a message based on commas and * which identifies the end of a NMEA line. I have also just recently tried the LINE INPUT command as well. Here is a sample of what is going on $GPRMC,234336.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*70 $GPRMC,234337.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*71 $GPRMC,234P338.000,A,2720.2571,S,15302.5943,E,0.00,183.56,20 1112,,,A*7E $GPRMC,234339.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*7F $GPRMC,234340.000,A,2720.2571,S,15302.5943,E,0.00,183.56,201 112,,,A*71 See the extra "P" snuck in there somehow. I tried playing with the serial buffer - did not really help. Is it too much for the DuinoMite to read NMEA, print to VGA and save to SD ? I have after about 12 hours, an error and the system stops writing to the SD card. I have tried opening the file, and leaving it open and just append - I lost a whole lot of data after a power failure - the file was not closed off. I also tried each NMEA line, writting to the SD card. open append and close. Am what I am doing a big ask ? I did everything in assembly language besides write to an SD CARD then I went looking and found DuniMite. The NMEA is from a MOD-GPS at 19200. I have a buffer of serial of 1024. I am keen to get this stable as it is great solution. Its just these two odd behaviours I can't explain The power supply is fine. I am printing to VGA the NMEA as well Here is the code, its pretty simple and displays odd charcaters and bombs out on SD card writes now and then file$ = "GPSLOG.LOG" Open "COM3:19200,1024" As #1 Do Line Input #1,msg$ Print msg$ Open file$ For append As #3 Print #3, MSG$ Close #3 Loop EXT is used - DuinoMite Mini - Andrew VK4TEC - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Here is the other way I do it Get character by character from the serial port // file$ = "GPSLOG.LOG" ' initial filename max =20 Dim arg$(max) Open "COM3:19200,1024" As #1 Open file$ For append As #3 Do nmea_sentence 'Print msg$ x$ = Input$(1, #1) msg$ = msg$ + x$ x$ = Input$(1, #1) msg$ = msg$ + x$ If arg$(0) = "GPRMC" Then Print msg$ file$ = arg$(9) + ".LOG" GoTo savefile EndIf If arg$(0) = "GPGGA" Then Print msg$ GoTo savefile EndIf Rem save to SD card GoTo andrew savefile: 'file$ = arg$(9) + ".LOG" 'Open file$ For append As #3 Print #3, MSG$ 'Close #3 andrew: Loop Rem -- SUBS -- Sub nmea_sentence Do msg$ ="$" ' subroutine start Do While Input$(1, #1) <> "$" : Loop ' wait for the start For i = 0 To max arg$(i) = "" ' clear ready for data Do ' loops until a specific exit x$ = Input$(1, #1) ' Print x$ msg$ = msg$ + x$ ' get the character If x$ = "," Then Exit ' new data field, increment i If x$ = "*" Then Return ' we have all the data so return arg$(i) = arg$(i) + x$ 'Print arg$(i) ' add to the data Loop 'Print arg$(i) ' loop back for the next char Next i ' move to the next data field ' Print "Corrupt data..." ' exceeded max data items 'Print msg$ Loop End Sub Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
I have tried something 1. Reduce the GPS rate to 9600 2. Turn off all but $GPRMC and $GPGGA 3. Gone back to the character by character mode - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
cwilt Senior Member Joined: 20/03/2012 Location: United StatesPosts: 147 |
Andrew, Have you tried a different GPS? I had a Duinomite Mega and a GPS shield from Sparkfun using similar software design running for months without issue. This is the shield. Does the error happen at completely random times? |
||||
Talbit Senior Member Joined: 07/06/2011 Location: AustraliaPosts: 210 |
Andrew, See my article on my GPS clock somewhere under my name. I'm using a Garmin GPS 16HVS running at 19200, 1Hz spitting out $GPRMC only. I haven't spotted any problems. Running at that speed have you got good layout with short interconnecting wires? I'm using the original Maximite with around v3.1 I think. Talbit Talbit |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Got some GPS I can try GPS18LVC GPS18 5Hz GPS72 Speed changes did not make a dif, maybe it is GPS - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Change of pace Dumped the MOD-GPS Hooked up GARMIN GPS185Hz Locked it down to 19200 Locked it down to $GPRMC So far so good Maybe GPS was cactus ? - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
I have now worked out my GPS issues. I now need to attend to the SD CARD issue. After a long time. The program will stop at open file$ for append as #3 Error: Cannot write to file. Does that mean the file is not closed off properley ? Is it too big ? Is it that I need some pause statements ? The SD card is written to many times. and quickly. - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Is there such a thing as software commanded reboot ? A software watchdog timer ? - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
bigmik Guru Joined: 20/06/2011 Location: AustraliaPosts: 2914 |
You could always try connecting an unsed ouptut pin (pulled high via a 10k or 4k7 OHM resistor) to the RESET.. when you want to reset the MM direct the pin to go LOW and bingo it kicks itself up the ..... errr ... well it resets itself. The MM should reconfigure the pin as an INPUT and the pull up would keep it high until the MM has control again. Regards, Mick Mick's uMite Stuff can be found >>> HERE (Kindly hosted by Dontronics) <<< |
||||
James_From_Canb Senior Member Joined: 19/06/2011 Location: AustraliaPosts: 265 |
Are we talking about the code you posted above? That code only opens the file before the main DO statement, which is the first few fractions of a second of processing. You should never get that message a long time after you've started the program. Would you please post your current code? Thanks James My mind is aglow with whirling, transient nodes of thought careening through a cosmic vapor of invention. Hedley Lamarr, Blazing Saddles (1974) |
||||
TinkersALot Regular Member Joined: 20/11/2012 Location: United StatesPosts: 72 |
How large is the file when the program quits? |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
I think I might have fixed it Instead of one big file or daily files, I have gone to hourly files. Has not bombed out yet. - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
The files at the moment are about 2 MB - Andew - Andrew Rich VK4TEC www.tech-software.net |
||||
bigmik Guru Joined: 20/06/2011 Location: AustraliaPosts: 2914 |
Thats Interesting Andrew, Maybe you have your SD card formatted at FAT12 which has a maximum file size of 32MB (which at 2MB per hr is about 16hrs use..) If you formatted with FAT16 you can get 2GB files or FAT32 you can get 4GB files.. Regards, Mick Mick's uMite Stuff can be found >>> HERE (Kindly hosted by Dontronics) <<< |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
What is odd, is when the program bombs out, I can restart it and it keeps going for a bit more. Its not as if it reaches a limit and then stops all together Maybe it is like the variable to write to the card or the offset for the SD card becomes too big. It sometimes bombs out trying to open, as if it did not close properley. Well at the moment it has not missed a beat at 2 MB hourly files. - Andrew - Andrew Rich VK4TEC www.tech-software.net |
||||
vk4tec Senior Member Joined: 24/03/2012 Location: AustraliaPosts: 239 |
Andrew Rich VK4TEC www.tech-software.net |
||||
MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
How about compressing/filtering the data. That would save a good amount of data. An entry i save is like this: Timestamp,longitude,latitude,direction,speed some real data: 19f2wq,4.516478,51.812278,N,0 The timestamp is compressed by using an offset and base64 coding of the timestamp. It is about 32 bytes on average. If you have 1 per second an hour would be about 32 * 60 * 60 = 115200 bytes. You can have about 17 hours worth of data in that 2 MB. I found for vehicle tracking once a minute is sufficient while driving and every 15 minutes while parked. It is really a datasaver. A vehicle transmitting that data over GPRS is on average 1.2MB for a month! All application differ but they do have in common to find out what data is really needed, not what is available as that often is just too much and can slow down the processing of that data a lot. Microblocks. Build with logic. |
||||
Print this page |