 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : Maximum File Size on SD Card and File Handling
| Author | Message | ||||
| stef123 Newbie  Joined: 25/09/2024 Location: United KingdomPosts: 23 |
Hi, i´ve implemented for my Webmite-program a subroutine, which is called each two seconds by a timer, in order to write some Log Data to a File on SD Card, which is periodically opened in "append" mode. Some Data is written (approx 20 Bytes), after that the File I/O Buffer is flushed in order to be sure that no data gets lost and the file is finally closed. Now, after the log File has reached a Size of 8339454 Bytes, i cannot write into this file any longer. Is this a general PicoMite File Size limit, or could it be that the FAT-Routines now consume too much time in order to find the end of the File and append some data? Is the PicoMite / are the FAT Routines in append mode in need to crawl through the entire file before some Data can be added - by following the clusters and read in the Data for each Cluster- or do the Fat routines seek for the last Cluster of the File in question and read it in? The next problem is reading the Log File from SD card over TFTP after the Pico has been restarted. Issuing a "tftp -i xx.xx.xx.xx get b:/log.txt" gives an empty file, although the Pico says "TFTP request to read binary file : b:/log.txt" and ends with "TFTP transmit complete" - but with no Data transmitted at all. The file can be only read entirely when the drive is changed from A to B by issuing "B:" on the command line. Is there any workaround for this behaviour? Thanks & Best regards Stef Edited 2024-10-30 17:46 by stef123 |
||||
| matherp Guru  Joined: 11/12/2012 Location: United KingdomPosts: 9084 |
Can't comment on the issue with the large file but you may well be pushing FatFS beyond its limits. The TFTP issue is a bug which I'll fix - thanks for the report |
||||
| stef123 Newbie Joined: 25/09/2024 Location: United KingdomPosts: 23 |
Hi Peter, thanks for you reply. So i assume it would be better to write logs individually on a daily basis, instead of pushing it all into a single file. It would have been a bit easier for the log analyzing program i´ve wrote on the PC, but that can be solved. |
||||
| Michal Senior Member  Joined: 02/02/2022 Location: PolandPosts: 123 |
32 bit overflow ? Overflow |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
8339454 is a bit under 8MB (8*1024*1024 = 8388608). A bit of an odd number. John |
||||
Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 6761 |
I'd be interested to know the actual format of the SD card - number of tracks/sectors/cluster sizes. Just saying "Fat32" or whatever makes assumptions. I wonder if the difference in number is because some of the space on the card is reserved for bad sector mapping? Edited 2024-10-30 21:01 by Mixtel90 Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
| stef123 Newbie Joined: 25/09/2024 Location: United KingdomPosts: 23 |
fsck.fat 4.2 (2021-01-31) Checking we can access the last sector of the filesystem There are differences between boot sector and its backup. This is mostly harmless. Differences: (offset:original/backup) 65:01/00 Not automatically fixing this. Boot sector contents: System ID " " Media byte 0xf8 (hard disk) 512 bytes per logical sector 32768 bytes per cluster 1170 reserved sectors First FAT starts at byte 599040 (sector 1170) 2 FATs, 32 bit entries 3894784 bytes per FAT (= 7607 sectors) Root directory start at cluster 2 (arbitrary size) Data area starts at byte 8388608 (sector 16384) 973584 data clusters (31902400512 bytes) 63 sectors/track, 255 heads 8192 hidden sectors 62325760 sectors total Totally normal 32 GB SD Card (formatted as FAT32). Edited 2024-10-30 22:27 by stef123 |
||||
| stef123 Newbie Joined: 25/09/2024 Location: United KingdomPosts: 23 |
Usually the Fat32-Limit is far beyond 32 GB. As with my own experiences with programming a FAT32 Driver (for 8051) and by that handling SD Cards on command level, i cannot imagine any reason aside from Speed issues why the writing gets stuck. I'll try to repeat the process without involving the Log program at all. Edited 2024-10-30 22:35 by stef123 |
||||
| Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 6761 |
Interesting. 8339454 / 32768 The error occurs at almost bang on 254.5 clusters (254.4999). I wonder if that's significant, like an 8-bit cluster counter? Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
| stef123 Newbie Joined: 25/09/2024 Location: United KingdomPosts: 23 |
Cannot repeat the issue for now, sorry. Writing a file with the same amount of data for each call of the subroutine and by the same append method creates a file larger than the size in question. I also cannot see any performance drop at all. I´ll let it run over night to see if any errors occur. Maybe this happens if anything other is involved. My program has also an RX-Interrupt involved. Edited 2024-10-30 23:36 by stef123 |
||||
| Mixtel90 Guru Joined: 05/10/2019 Location: United KingdomPosts: 6761 |
Probably POM dependent. :) Mick Zilog Inside! nascom.info for Nascom & Gemini Preliminary MMBasic docs & my PCB designs |
||||
 Print this page |
