 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : WebMite: How to read enegyprice from a website
Page 1 of 2   |
|||||
| Author | Message | ||||
| Pluto Guru  Joined: 09/06/2017 Location: FinlandPosts: 357 |
I have to renew my energy supply contract soon. The new contract will be SPOT based and suddenly it becomes interesting to follow the electricity price hour by hour and also to check prices for the next day. This could in the simplest way be done on the PC or with phone apps, but much more interesting to make use of one the WebMites laying around my desc. I have found some (hopefully) useful sites with price info for the Finnish electricity market. One of them is called "sahkohinta-api.fi". Unfortunately only in finnish. On Google Chrome you just have to enter: https://www.sahkohinta-api.fi/api/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15 to find the cheapest hour in the day. The reply looks like this [{"aikaleima_suomi":"2024-11-15T23:00","aikaleima_utc":"2024-11-15T21:00","hinta":"-0.08700"}] (yes; negative price tonight between 23:00-23:59). Timestamp FI, Timestamp UTC, hinta=price (eurocents/kWh)."sahkohinta-api.fi" is a free service. No need to register and you can use it 60 times per hour max. There are a few different "api-requests": a. Search for the cheapest hour b. Search for the most expensive hour c. Search for the cheapest hours (you define how many to list) d. Search for the most expensive hour (you define how many to list) I have tried to do this on a WebMite by using WEB TCP CLIENT without success. Without sufficient knowledge I have tried several times by trial and error (and only error) like this OPTION EXPLICIT OPTION DEFAULT NONE Dim buff%(4096/8) DIM A% = 1000 'Dim b$= "GET /api/v1/halpa?tunnit=[2]&tulos=haja&aikaraja=[2024-11-15] HTTP/1.1\r\nhost:sahkohinta-api.fi\r\n\r\n" 'Dim b$= "GET /api/v1/halpa?tunnit=[2]&tulos=haja&aikaraja=[2024-11-15] HTTP/1.1\r\nhost:sahkohinta-api.fi" 'dim b$="GET / HTTP ap1/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15"+Chr$(13)+Chr$(10) dim b$="GET ap1/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15" WEB open TCP CLIENT "SAHKOHINTA-API.FI",80 WEB TCP CLIENT REQUEST b$,buff%(),10000 'web TCP SEND 1,b$ 'web tcp read 1,buff%() 'web tcp read 1,buff%() WEB close TCP CLIENT 'a%=LinStr(buff%(),"{")-1 'If a% <> -1 Then 'LongString trim buff%(),a% LongString print buff%() Print Json$(buff%(),"hinta") Connected [10] WEB TCP CLIENT REQUEST b$,buff%(),10000 Error : No response from server > option list WebMite MMBasic Version 6.00.00RC15 OPTION SYSTEM SPI GP18,GP19,GP16 OPTION LCDPANEL ILI9341, LANDSCAPE,GP15,GP14,GP13 OPTION WIFI HuaHuaNisse, ***********, PICOE6616408432 OPTION TCP SERVER PORT 80 OPTION TOUCH GP12,GP11 GUI CALIBRATE 0, 190, 227, 909, 668 I am not sure if the TCP is the correct route. What else could be tried? Is there any way to see what Chrome is actually sending when you issue the command in the browser? I briefly read the lectures by Luc again, but for this case I did not find any hints. Should maybe read more carefully. Any help and ideas are appreciated. Thanks, Pluto |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I used telnet sahkohinta-api.fi 80 and simplified it to GET /api/v1/halpa HTTP/1.1 (and CR LF) host:sahkohinta-api.fi (and CR LF) then a blank line which is another \r\n (CR LF) And got 35 {"virhe" : "pakollinen hakuparametri puuttuu: tulos"} 0 Maybe gives a clue? John |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
Thanks John! It gives several clues...TELNET instead of TCP Client... You got at least an answer! Even if it is an error message: translated "Error: mandatory search parameter missing: result" Thanks a lot. Now I can continue in a more promising direction. Fred |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I wonder if you may need to encode some "special" chars e.g. = to %3D and/or [ to %5B Or use HTTP-type encoding such as & to & amp ; John Edited 2024-11-15 22:49 by JohnS |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
John I am a bit confused re your TELNET advice. Do you do it directly in the MMCC console? I tried but with no luck. Fred |
||||
| Volhout Guru Joined: 05/03/2018 Location: NetherlandsPosts: 4212 |
I am very new to this, but I never used port 80 for telnet. Telnet is typically port 23. Time for me to pay attention and start learning from the seniors. I will follow this thread with care... Volhout PicomiteVGA PETSCII ROBOTS |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I did it from a command line (Linux; would work in Windows if you have telnet or install a telnet client). The reason it works at all is many net protocols are ASCII-type stuff (*) once you're connected. You connect at a host (domain name etc) and port (23 if you want telnet itself, 25 for SMTP, 80 for http, usually, and so on). (*) not very efficient but such a great thing for debugging!! (Thanks go to the net pioneers who made this stuff much easier than binary would've been.) John Edited 2024-11-16 02:57 by JohnS |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
John, not much progress here. I have installed TELNET (win 10) and got a connection to sahkohinta-api.fi and got this in the cmd window Microsoft Telnet> open sahkohinta-api.fi SSH-2.0-mod_sftp ☻õ♦¶°Ë;┼↓♦°▄Ô,Ô‗w ☺ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group18-sha512,diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,rsa1024-sha1,ext-info-s)rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dssVaes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,cast128-cbc,3des-cbcVaes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,cast128-cbc,3des-cbc[hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,umac-64@openssh.com,umac-128@openssh.com[hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha1-96,umac-64@openssh.com,umac-128@openssh.com♦none♦none▲¼çB I do not understand the content of it. This stays open a while (connection open?), but I have no idea about how to continue with your proposed Fred |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
Looks like it's not using port 80, so not using http. Did you put 80 as the port? It's not the default. John |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
Opened: open sahkohinta-api.fi 80 GET /api/v1/halpa HTTP/1.1 => 400 Bad request or 404 Not found Years since I last opend the CMD. Very messy output! Have to stop for today. Thanks for your advice! Fred |
||||
palcal Guru Joined: 12/10/2011 Location: AustraliaPosts: 1873 |
If you don't mind me asking, what is the cost of energy in Finland. Where I am in Australia I pay 30 cents per kilowatt hour. I do have solar panels which offset the cost somewhat. "It is better to be ignorant and ask a stupid question than to be plain Stupid and not ask at all" |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
Spot priced energy is from 0 (night) to about 30 €cents/kWh (peak). Charged per hour. I think it is possible to get an average of <5. Fixed priced contracts are about 9...15 cents/kWh. On top of these comes the transfer price about 5 cents/kWh. The transfer prices are increasing rapidly as there is no competition between the transfer companies. Each region is covered by one transfer company.  Fred |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 9084 |
Your original post shows google connecting over HTTPS. Webmite only supports HTTP. You will need to see if the supplier has an api available over http. If not you are probably stuck. |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
Thanks Peter. You are correct; I am stuck.  I will try to look for an other data supplier. Fred |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
It seems that all potentially useful datasuppliers I have found use HTTPS. Its a pity that WebMite can't work with HTTPS. I suppose there is not any work-around to get WebMite to work with HTTPS. Should it be better to try with ESP8266 or ESP32 and Annex? Do they manage with HTTPS? (My previous experience with ESP8266 is not very positive...) Maybe I have to skip the whole idea about an on-line PicoMite price-indicator. Had also planned to have a price based controller for the water boiler. I think that the idea has to rest until WebMite is HTTPS-compatible. Thanks for help and comments! Fred |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I apologise for not mentioning it but I changed https to http and it worked fine. All my subsequent tests have been http. I'm missing some magic thing to make them work fully. (Maybe Accept stuff?) John |
||||
| Pluto Guru Joined: 09/06/2017 Location: FinlandPosts: 357 |
Hi John, so there might be light in the tunnel? Hope so! |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
Yes. It works via http with tools such as wget (likely also curl) as well as a browser (I used firefox), so it almost for sure can work using a webmite. The problem seems to be either the exact URL sent or the extra (header) lines sent before the (empty) message body. Maybe nc (netcat) can help but I've not used it to debug stuff like this. I once wrote (not in C lol) simple versions of smtp, smtpd, pop3, pop3d and who knows what but never http and it was all 16-bit code back then, not even on Linux. I'll mess around a bit more when I get time. John |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I turned on some debugging: wget -d -v -S 'http://www.sahkohinta-api.fi/api/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15' It sent GET /api/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15 HTTP/1.1 Host: www.sahkohinta-api.fi User-Agent: Wget/1.21.4 Accept: */* Accept-Encoding: identity Connection: Keep-Alive Got back HTTP/1.1 200 OK Date: Sat, 16 Nov 2024 13:39:15 GMT Server: Apache/2.4.57 (Unix) OpenSSL/3.0.2 X-Powered-By: PHP/7.4.33 Upgrade: h2 Connection: Upgrade, Keep-Alive Keep-Alive: timeout=5, max=100 Transfer-Encoding: chunked Content-Type: application/json Data it got [{"aikaleima_suomi":"2024-11-15T23:00","aikaleima_utc":"2024-11-15T21:00","hinta":"-0.08700"}] You are unlikely to need to send the User-Agent. Probably don't need the Keep-Alive. Not sure about the Accept-Encoding: identity John Edited 2024-11-16 23:45 by JohnS |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3797 |
I did a telnet test and it worked: telnet sahkohinta-api.fi 80 GET /api/v1/halpa?tunnit=1&tulos=haja&aikaraja=2024-11-15 HTTP/1.1 Host: www.sahkohinta-api.fi Accept: */* Note the final blank line to show end of headers and no message body. John |
||||
| Page 1 of 2 |
|||||
 Print this page |
