 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : Wattmon uPHP on PIC32
Page 1 of 3   |
|||||
| Author | Message | ||||
| akashh Senior Member  Joined: 19/01/2014 Location: IndiaPosts: 115 |
Hello Everyone! This is my first post on this forum although I have been following it for a while, and I would like to say that it is great reading with lots of good ideas and info! I have spent the last year and a half developing an embedded scriptable OS which I call Wattmon and in my research on other interpreted languages running on the pic32 I discovered the maximite and MMbasic. Although my product is commercial, aimed at solar and renewable energy monitoring, I am thinking to release a pre compiled hex version to run on a maximite or other pic32 board. Wattmon runs an RTOS and requires a microSD card and Ethernet interface to work right now. It runs a web server and allows for dynamic pages to be generated using a subset of the PHP language. It also has a scheduler and can run scripts up to once a second if required. I have developed an application that runs off the sd card which allows you to manage files and edit scripts using a powerful editor right in your browser. It has a telnet shell that lets you log in remotely and run scripts as well. The firmware also has a usb host driver for USB 3G dongles, which will connect you to the net with a dynamic IP. The hardware version I developed has a modbus RTU interface to various sensors. It has functions to read analog inputs, communicate via SPI, onewire and serial, and I am adding I2C now. I can think of a million applications for this product but since it is relatively new and as of yet unknown, I would be ready to try and adapt parts of it to run on a different platform if any of you would like to play with it and give me feedback. I have some info up on www.wattmon.com already. I look forward to comments. Cheers Akash  |
||||
Sumit Newbie  Joined: 13/09/2011 Location: Posts: 23 |
This looks interesting. I would like to know more.  |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Sumit, Thanks for the interest. I just purchased a Duinomite and UEXT ethernet adapter from Olimex and I guess the first step is to port it to that platform, so you could download and install it. Which platform are you currently using? |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Ok I got my duinomite last week, and of course the first thing was to solder on some wires since the icsp connector does not match the pickit3... That went smoothly. Now I am stuck, I ported my sd bootloader which is a slightly modified version of microchips bootloader and modified the SPI port and LEDs. It now detects the sd card and tries to find the hex file but seems to hang somewhere between loading the fat and finding the file, whereas this works fine on my dev board (with different pin mappings) (which is hand soldered). Did anyone experience issues with sd communication or is it just something I have not found yet? That would be the first part required for me to port the WattmonOS to duinomite. |
||||
| Sumit Newbie Joined: 13/09/2011 Location: Posts: 23 |
Akashh, Sorry for the late response. I do not use any specific platform. I always start with bare PICs with the required firmware on my breadboard. So, I am happy to work with any setup/platform. |
||||
| Sumit Newbie Joined: 13/09/2011 Location: Posts: 23 |
I have had some issues with the SD card in the past, but they were more due to the incompatible SD-Card rather than the design or the board. |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Sumit, it isn't an SD card issue since the same card works on my board. I will dig further and let you know. Akash |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Here is a picture of the Wattmon board with USB dongle plugged in an connected to the Internet. The RTC and battery backup keeps track of time. The HV power supply can be seen at one end. microSD, modbus port (RJ45), Ethernet and USB are all along one edge. 
|
||||
Geoffg Guru  Joined: 06/06/2011 Location: AustraliaPosts: 3196 |
Hi Sumit, good to see you back. Geoff Geoff Graham - http://geoffg.net |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Guys, I am having an occasional problem with some of my microSD cards. They are formatted as FAT32 (4KB block size). After several months, they seem to stop working properly. I am logging to disk once a minute. One file is overwritten every minute, while another couple of files are appended to. When I put the card into a PC (linux), I can see the folder structure but when I try copying out, after a few files the card disappears, giving an error. Strangely, I can sometimes recover the files by reading them one by one through the Wattmon web interface. Formatting the disk under windows sometimes solves the problem, and other times the disk is just bricked. I am using SandDisk, which has quite some documentation and they claim they use wear leveling at the sector level, which means that this should not be causing a problem for a microcontroller. Sometimes this is caused by a FAT corruption which can be recovered with a disk check, but it is not necessary related. My question is this: Could this be related to wearing out of the flash, or would it be more likely to be related to a corruption during writing to the sd? Also, does anyone know whether it is better to delete and create a new file (small files, less than 4 kb) every time, or is it better to simply rewrite an existing file for long term performance? Any tips would be helpful. Akash |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3196 |
By my calculations you would have written to the card a total of 86,000 times over three months and each one would have updated the FAT. My understanding is that most low cost cards have an endurance of 20,000 to 40,000 erase/writes while SanDisk claim 100,000. Maybe it is a SanDisk counterfeit? Regardless, writing once a minute is asking for trouble. Geoff Geoff Graham - http://geoffg.net |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Thanks Geoff, I have read whichever white papers I could get my hands on and concluded that Sandisk's low level wear leveling would be able to handle plenty of writes since it would each time place it in a different part of the cluster, it seems to have some type of invisible internal mapping table. The reality is that things seems to stop working over a while. But strangely some of my devices are going strong after a year, and still working OK while others fail after a quite short interval. I am purchasing my cards through a reputable dealer so I doubt they are counterfeit. I guess I could write once every 5 or 10 minutes instead. What's your take on whether it's better to overwrite the same file or use a new file every time (and then delete the old one)? The upside is that it could continuously use new sectors, the down side is that the FAT table maybe become extremely large, I think somehow old deleted entries are still kept in the FAT, aren't they? Akash |
||||
| Geoffg Guru Joined: 06/06/2011 Location: AustraliaPosts: 3196 |
I don't think that it makes much difference appending or creating a new file. They will all update the FAT and I bet that is where the wear is being felt. Yes, deleted entries are kept in the FAT but are eventually overwritten and reused. Why not buffer the data and then write it once an hour? The your cards would last for 10 years or more. If you are worried about power failure you could use some large caps in the power supply and do a quick write/close when the falling source voltage is detected. Geoff Geoff Graham - http://geoffg.net |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
I want a minute-by-minute log of various parameters and I think 60 entries in ram would take up too much space (I have a variable number of parameters that need to be logged, sometimes as much as 20) but perhaps 10 minutes would be doable. Thanks for the advice. Have you had any issues with SD degradation on MMBasic? |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Here's a screen shot of the latest web interface dashboard running on Wattmon. 
Sorry, for some reason it is very small, and when I link the large one, it changes the format of the page too much, so here's a link if you are interested: Link to bigger image |
||||
MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
FRAM is the best solution. Just keep your logs in that and when it becomes full or once an hour/day whichever is more convenient and write it to an SD. This would also allow the user to remove the SD to copy data on his pc while the log is still being appended to in the FRAM. Microblocks. Build with logic. |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
When designing a new solution that could be good, but I have 100 boards assembled and ready, so I need to figure out something that will work with the ram I have. Also, removing the sd while running isn't an option since scripts actually run off the sd card if they cannot fit in the available ram. There is a small speed penalty but this allows for very large scripts, even hundreds of k, to be run if required. An example of this would be a web page with lots of HTML and some embedded scripts here and there. Anyway, the discussion is about writing and not reading. I guess since I need to write csv data, storing them in a string would be the most efficient way, and could theoretically grow to 1 or 2 k without creating too much impact on available ram. Another and perhaps better way would be to sync to disk whenever ram is required, which is actually only when someone is browsing the web server. Thanks for the suggestions so far, if anything else comes to mind please share. Edit: scripts are tokenised if the source file has been modified, and then loaded into ram buffers of 1k each when a script is run, so most basic scripts that run once a second or so can be fully buffered in ram. The oldest cache buffer is automatically reused, so the most recently used parts of a program will always be in ram, which will speed up loops or function calls to small functions. |
||||
| MicroBlocks Guru Joined: 12/05/2012 Location: ThailandPosts: 2209 |
if already commited to hardware the solution has to be in the software. One way of minimising the wear would be to modify the fat driver layer. if every open/write/close will also modify the fat tables then wear will render the sd card useless. what you could do is create a file of a big enough size to hold your data, fill it with zeros. use open for append, append as many zeros as is needed for as an example a day. then close the file. this will create a fat entry that registers which blocks are used. then write to these blocks directly, without going through the fat driver layer. accessing the sd card in spi mode would give you that possibility. i don't have any code, this is what i understand how an sd card is often used from a mcu. Microblocks. Build with logic. |
||||
| akashh Senior Member Joined: 19/01/2014 Location: IndiaPosts: 115 |
Thanks TZAdvantage, that makes a lot of sense. I am using spi mode and in microchips MDD library. I was thinking of switching to fatfs, butvibam not sure if there will be any major improvements. I read somewhere that people have experienced 200% or more increases in speed but I think that is only for sustained reades and writes of large amounts of data. Do you think that the typical 10k file would benefit from fatfs? I will certainly look into your idea of creating a large file. |
||||
| Sumit Newbie Joined: 13/09/2011 Location: Posts: 23 |
Hello Geoff, Yes, I have been off the forums for a long time. I have been busy with my work. Its exciting to know that you are coming up with new platform(s) |
||||
| Page 1 of 3 |
|||||
 Print this page |
