{"id":328,"date":"2017-06-02T20:55:03","date_gmt":"2017-06-02T10:55:03","guid":{"rendered":"http:\/\/bakke.online\/?p=328"},"modified":"2022-01-22T20:10:55","modified_gmt":"2022-01-22T09:10:55","slug":"self-updating-ota-firmware-for-esp8266","status":"publish","type":"post","link":"https:\/\/www.bakke.online\/index.php\/2017\/06\/02\/self-updating-ota-firmware-for-esp8266\/","title":{"rendered":"Self-updating OTA firmware for ESP8266"},"content":{"rendered":"

As part of my recent projects, I have started including OTA firmware updates for my ESP8266 devices. \u00a0(Also known as FOTA)<\/p>\n

Doing so is actually remarkably easy, thanks to the very good support for this exposed by the Arduino board support package. \u00a0The hardest thing actually becomes getting the web server side set up, rather than the changes required on the device itself.<\/p>\n

<\/p>\n

Firmware server<\/h3>\n

The firmware server can be any web server accessible over HTTP. (Or HTTPS, will get back to that later)<\/p>\n

\"\"<\/a><\/p>\n

For my projects, I decided to create a folder containing the firmware images, and to give each image a name derived from the MAC address of the ESP8266 device it applies to. \u00a0Along with each firmware image is a simple text file with a single line containing a build number\/version number.<\/p>\n

\"\"<\/a><\/p>\n

In my enviroment, I use a 32-bit integer to identify the build number. \u00a0Not because I believe I will have to release more than 65,536 firmware versions, but because that gives me more flexibility in how I format the build numbers, for example by representing build dates directly in the number. \u00a0However, for simplicity and the sake of this example, I’ll just put a plain version number in there.<\/p>\n

The specifics of how you organise and name the firmware images is completely up to you, of course. \u00a0For example, if you do larger production runs, you could use a device model number as the base for your firmware image naming convention instead.<\/p>\n

Publishing firmware images<\/h3>\n

Add a version number constant variable to your sketch, outside of any functions. \u00a0Every time you are ready to release, increase this version number by whatever convention you choose, for example a simple increase by one each time, or a more complex rule based on the current date and an increasing build number.<\/p>\n

const int<\/span> FW_VERSION = 1244;<\/pre>\n
Compile the sketch (Ctrl+R) and then export the binary. (Ctrl+Alt+S) \u00a0Exporting the binary will generate an image file into the same folder your sketch is in. \u00a0The actual name will depend on the particular board you are compiling for. \u00a0For the Adafruit Huzzah ESP8266<\/a> breakout, the name is the same as your main .ino file, but with .adafruit.bin appended.<\/p>\n
\"\"<\/a><\/p>\n
This file needs to be copied to the web server. \u00a0How you do this depends on the type of web server you are using. \u00a0I’m showing an example here, based on Apache2 running on the same computer I am using for compilation. \u00a0Your paths may vary depending on your operating system and web server.<\/p>\n
cp WifiTempMonitor.ino.adafruit.bin \/var\/www\/html\/fota\/5ccf7fXXXXXX.bin<\/pre>\n
Next, after the firmware has been published, update the version number in the version file and publish that as well. \u00a0This needs to be done after the firmware image, otherwise the wireless devices may see the updated version file and try to download a new firmware. \u00a0If your new firmware is not ready on the server at that time, the devices may get either an old version or an incomplete file. \u00a0An incomplete file could potentially brick your device…<\/strong><\/em><\/p>\n
Adding OTA support to the ESP8266 device<\/h3>\n
Start by adding #include statements for the OTA support library and the HTTP client library if you haven’t already included it.<\/p>\n
#include<\/span> <ESP8266HTTPClient.h>\r\n#include<\/span> <ESP8266httpUpdate.h><\/pre>\n
Add a global constant variable for the current firmware version.<\/p>\n
const int<\/span> FW_VERSION = 1244;<\/pre>\n
Next, we’ll need to tell the device where to find firmware updates. \u00a0We have both an image and a version file, organised by device MAC, so we’ll need to define a base URL that we can derive the actual URLs from.<\/p>\n
const char<\/span>* fwUrlBase = \"http:\/\/192.168.0.1\/fota\/<\/span>\";<\/pre>\n
Exactly where in your code you add the firmware check and update is up to you. \u00a0The only requirement is that for obvious reasons WiFi must be enabled and connected at the time. \u00a0In my weather station devices I make the check after all measurements have been taken and uploaded to the server, just before the device goes back to deep sleep. \u00a0We’ll define a function to check and update, and then just call it from a suitable location in your sketch.<\/p>\n
We’ll build the URL to download the version number of the currently available firmware image. \u00a0To get the device’s MAC address, call getMAC(), which will return it as a string of 12 hex digits.<\/p>\n
void<\/span> checkForUpdates() {\r\n  String<\/span> mac = getMAC();\r\n  String<\/span> fwURL = String<\/span>( fwUrlBase );\r\n  fwURL.concat<\/span>( mac );\r\n  String<\/span> fwVersionURL = fwURL;\r\n  fwVersionURL.concat<\/span>( \".version\" );\r\n\r\n  Serial<\/strong><\/span>.println<\/span>( \"Checking for firmware updates.<\/span>\" );\r\n  Serial<\/strong><\/span>.print<\/span>( \"MAC address:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( mac );\r\n  Serial<\/strong><\/span>.print<\/span>( \"Firmware version URL:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( fwVersionURL );<\/pre>\n
Download the version string and convert it to a 32-bit integer.<\/p>\n
HTTPClient httpClient;\r\nhttpClient.begin<\/span>( fwVersionURL );\r\nint<\/span> httpCode = httpClient.GET();\r\nif<\/span>( httpCode == 200 ) {\r\n  String<\/span> newFWVersion = httpClient.getString();\r\n\r\n  Serial<\/strong><\/span>.print<\/span>( \"Current firmware version:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( FW_VERSION );\r\n  Serial<\/strong><\/span>.print<\/span>( \"Available firmware version:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( newFWVersion );\r\n\r\n  int<\/span> newVersion = newFWVersion.toInt();<\/pre>\n
When connecting to a local firmware server, this check takes about 20 ms extra time and it uses a tiny bit of battery power. \u00a0By itself, this is less than the background noise in the energy readings, so I would not be too concerned with additional power consumption caused by the firmware updates.<\/p>\n
We compare the new firmware version number to the current version as defined in our global constant variable. \u00a0If the version available on the server is newer than what we’re running on, build the URL to download the firmware image from and pass this to the OTA updater.<\/p>\n
if<\/span>( newVersion > FW_VERSION ) {\r\n  Serial<\/strong><\/span>.println<\/span>( \"Preparing to update.\" );\r\n\r\n  String<\/span> fwImageURL = fwURL;\r\n  fwImageURL.concat<\/span>( \".bin<\/span>\" );\r\n\r\n  t_httpUpdate_return ret = ESPhttpUpdate.update<\/span>( fwImageURL );<\/pre>\n
The OTA updater downloads the image and writes it into the flash memory. \u00a0If the OTA updater is successful it will never return as it will reboot the ESP8266. \u00a0There could be reasons the updater will fail, so we have to be prepared to handle such failures. \u00a0The usual cause for failure would be that the firmware image file could not be downloaded for some reason, or that the connection was lost while downloading. \u00a0Such errors will not leave your device in an unusable state as the actual firmware flashing process would not have started yet.<\/p>\n
\u00a0 switch<\/span>(ret) {\r\n\u00a0 \u00a0 case<\/span> HTTP_UPDATE_FAILED:\r\n\u00a0 \u00a0 \u00a0 Serial<\/strong><\/span>.printf<\/span>(\"HTTP_UPDATE_FAILED Error (%d): %s<\/span>\", \u00a0ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());\r\n\u00a0 \u00a0 \u00a0 break<\/span>;\r\n\r\n\u00a0 \u00a0case<\/span> HTTP_UPDATE_NO_UPDATES:\r\n\u00a0 \u00a0 \u00a0 Serial<\/strong><\/span>.println<\/span>(\"HTTP_UPDATE_NO_UPDATES<\/span>\");\r\n\u00a0 \u00a0 \u00a0 break<\/span>;\r\n\u00a0 }\r\n}<\/pre>\n
The HTTP_UPDATE_NO_UPDATES case will not happen with the firmware server as described above. \u00a0It is used with a firmware server that can compare the current version with the available one. \u00a0I will get back to that later but you can remove those 3 lines from the code for now if you wish.<\/p>\n
I’ll repeat the whole function here along with a little bit extra debug output for easy copy and paste.<\/p>\n
void<\/span> checkForUpdates() {\r\n  String<\/span> mac = getMAC();\r\n  String<\/span> fwURL = String<\/span>( fwUrlBase );\r\n  fwURL.concat( mac );\r\n  String<\/span> fwVersionURL = fwURL;\r\n  fwVersionURL.concat( \".version<\/span>\" );\r\n\r\n  Serial<\/strong><\/span>.println<\/span>( \"Checking for firmware updates.<\/span>\" );\r\n  Serial<\/strong><\/span>.print<\/span>( \"MAC address:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( mac );\r\n  Serial<\/strong><\/span>.print<\/span>( \"Firmware version URL:<\/span> \" );\r\n  Serial<\/strong><\/span>.println<\/span>( fwVersionURL );\r\n\r\n  HTTPClient httpClient;\r\n  httpClient.begin<\/span>( fwVersionURL );\r\n  int<\/span> httpCode = httpClient.GET();\r\n  if<\/span>( httpCode == 200 ) {\r\n    String<\/span> newFWVersion = httpClient.getString();\r\n\r\n    Serial<\/strong><\/span>.print<\/span>( \"Current firmware version:<\/span> \" );\r\n    Serial<\/strong><\/span>.println<\/span>( FW_VERSION );\r\n    Serial<\/strong><\/span>.print<\/span>( \"Available firmware version:<\/span> \" );\r\n    Serial<\/strong><\/span>.println<\/span>( newFWVersion );\r\n\r\n    int<\/span> newVersion = newFWVersion.toInt();\r\n\r\n    if<\/span>( newVersion > FW_VERSION ) {\r\n      Serial<\/strong><\/span>.println<\/span>( \"Preparing to update<\/span>\" );\r\n\r\n      String<\/span> fwImageURL = fwURL;\r\n      fwImageURL.concat<\/span>( \".bin<\/span>\" );\r\n      t_httpUpdate_return ret = ESPhttpUpdate.update<\/span>( fwImageURL );\r\n\r\n      switch<\/span>(ret) {\r\n        case<\/span> HTTP_UPDATE_FAILED:\r\n          Serial<\/strong><\/span>.printf<\/span>(\"HTTP_UPDATE_FAILD Error (%d): %s<\/span>\", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());\r\n          break<\/span>;\r\n\r\n        case<\/span> HTTP_UPDATE_NO_UPDATES:\r\n          Serial<\/strong><\/span>.println<\/span>(\"HTTP_UPDATE_NO_UPDATES<\/span>\");\r\n          break<\/span>;\r\n      }\r\n    }\r\n    else <\/span>{\r\n      Serial<\/strong><\/span>.println<\/span>( \"Already on latest version<\/span>\" );\r\n    }\r\n  }\r\n  else <\/span>{\r\n    Serial<\/strong><\/span>.print<\/span>( \"Firmware version check failed, got HTTP response code<\/span> \" );\r\n    Serial<\/strong><\/span>.println<\/span>( httpCode );\r\n  }\r\n  httpClient.end<\/span>();\r\n}<\/pre>\n
This code relies on a getMAC() function.\u00a0 I had originally missed this from the post.\u00a0 Thanks to Akshay for pointing that out, and to Mike Kranidis for pointing me to sprintf(), which is supported on the ESP.<\/p>\n
String<\/span> getMAC()\r\n{\r\n  uint8_t<\/span> mac[6];\r\n  char <\/span>result[14];\r\n\r\n snprintf( result, sizeof<\/span>( result ), \"%02x%02x%02x%02x%02x%02x\"<\/span>, mac[ 0 ], mac[ 1 ], mac[ 2 ], mac[ 3 ], mac[ 4 ], mac[ 5 ] );\r\n\r\n  return<\/span> String<\/span>( result );\r\n}<\/pre>\n
Providing current firmware version number<\/h3>\n
It is quite useful to know which firmware version is currently running on each device. \u00a0This is typically shown on a web page provided by a built-in web server on the ESP8266 device, or by debug output on the serial port. \u00a0But how about when you are dealing with battery powered devices which spend the vast majority of their time in deep sleep? \u00a0There would be no serial or WiFi connection up and running for you to check the firmware version.<\/p>\n
As my devices already use MQTT to transmit their sensor readings to a central server, I just define an additional channel for the firmware version. \u00a0(This code assumes you have already configured and connected your MQTT connection)<\/p>\n
Adafruit_MQTT_Publish<\/strong><\/span> publish_version = Adafruit_MQTT_Publish<\/strong><\/span>( &mqtt, MQTT_USERNAME \"\/feeds\/<\/span>\" MQTT_NAME \"\/fw_version<\/span>\", MQTT_QOS_0, 1 );\r\n\r\n\r\n publish_version.publish<\/span>( FW_VERSION );<\/pre>\n
The last parameter to the publisher constructor above is a flag to tell MQTT the message should be retained. \u00a0For more details, see one of my earlier posts<\/a> on how to add this functionality.<\/p>\n
Future improvements<\/h3>\n
There are areas that can be improved in the update process I have described.<\/p>\n
HTTPS support<\/strong> will help in keeping the firmware images encrypted in transit and also reduce the risk of compromised firmware images being provided by someone intercepting the traffic between your ESP8266 device and the firmware server.<\/p>\n
A smarter firmware server<\/strong> can also be built to avoid the need for a separate version file. \u00a0However, while this simplifies the code on the device a bit, it also adds quite a bit to the complexity of the server side.<\/p>\n
References<\/h3>\n
A description of the OTA process can be found at\u00a0http:\/\/esp8266.github.io\/Arduino\/versions\/2.0.0\/doc\/ota_updates\/ota_updates.html<\/a>\u00a0 This page also describes other ways of updating the firmware, including on-demand.<\/p>\n","protected":false},"excerpt":{"rendered":"
As part of my recent projects, I have started including OTA firmware updates for my ESP8266 devices. \u00a0(Also known as FOTA) Doing so is actually remarkably easy, thanks to the very good support for this exposed by the Arduino board support package. \u00a0The hardest thing actually becomes getting the web server side set up, rather … Continue reading “Self-updating OTA firmware for ESP8266”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7,25,1],"tags":[8,9,28,23,24],"class_list":["post-328","post","type-post","status-publish","format-standard","hentry","category-arduino","category-the-notwork","category-uncategorized","tag-arduino","tag-esp8266","tag-fota","tag-internet-of-things","tag-notwork"],"_links":{"self":[{"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/posts\/328","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/comments?post=328"}],"version-history":[{"count":3,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/posts\/328\/revisions"}],"predecessor-version":[{"id":1315,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/posts\/328\/revisions\/1315"}],"wp:attachment":[{"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/media?parent=328"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/categories?post=328"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.bakke.online\/index.php\/wp-json\/wp\/v2\/tags?post=328"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}