esp32/README: Update based on new IDF v4 cmake build process.

Signed-off-by: Damien George <damien@micropython.org>

1 files changed, 76 insertions, 169 deletions

diff --git a/ports/esp32/README.md b/ports/esp32/README.md
index 54fb41cf6..4f6375059 100644
--- a/ports/esp32/README.md
+++ b/ports/esp32/README.md
@@ -1,13 +1,13 @@
 MicroPython port to the ESP32
 =============================
 
-This is an experimental port of MicroPython to the Espressif ESP32
-microcontroller.  It uses the ESP-IDF framework and MicroPython runs as
+This is a port of MicroPython to the Espressif ESP32 series of
+microcontrollers.  It uses the ESP-IDF framework and MicroPython runs as
 a task under FreeRTOS.
 
 Supported features include:
 - REPL (Python prompt) over UART0.
-- 16k stack for the MicroPython task and 96k Python heap.
+- 16k stack for the MicroPython task and approximately 100k Python heap.
 - Many of MicroPython's features are enabled: unicode, arbitrary-precision
   integers, single-precision floats, complex numbers, frozen bytecode, as
   well as many of the internal modules.
@@ -15,16 +15,24 @@ Supported features include:
 - The machine module with GPIO, UART, SPI, software I2C, ADC, DAC, PWM,
   TouchPad, WDT and Timer.
 - The network module with WLAN (WiFi) support.
+- Bluetooth low-energy (BLE) support via the bluetooth module.
 
-Development of this ESP32 port was sponsored in part by Microbric Pty Ltd.
+Initial development of this ESP32 port was sponsored in part by Microbric Pty Ltd.
 
-Setting up the toolchain and ESP-IDF
-------------------------------------
+Setting up ESP-IDF and the build environment
+--------------------------------------------
 
-There are two main components that are needed to build the firmware:
-- the Xtensa cross-compiler that targets the CPU in the ESP32 (this is
-  different to the compiler used by the ESP8266)
-- the Espressif IDF (IoT development framework, aka SDK)
+MicroPython on ESP32 requires the Espressif IDF version 4 (IoT development
+framework, aka SDK).  The ESP-IDF includes the libraries and RTOS needed to
+manage the ESP32 microcontroller, as well as a way to manage the required
+build environment and toolchains needed to build the firmware.
+
+The ESP-IDF changes quickly and MicroPython only supports certain versions.
+Currently MicroPython supports v4.0.2, although other IDF v4 versions may also
+work.
+
+To install the ESP-IDF the full instructions can be found at the
+[Espressif Getting Started guide](https://docs.espressif.com/projects/esp-idf/en/v4.0.2/get-started/index.html#installation-step-by-step).
 
 If you are on a Windows machine then the [Windows Subsystem for
 Linux](https://msdn.microsoft.com/en-au/commandline/wsl/install_guide) is the
@@ -32,164 +40,31 @@ most efficient way to install the ESP32 toolchain and build the project. If
 you use WSL then follow the Linux instructions rather than the Windows
 instructions.
 
-The ESP-IDF changes quickly and MicroPython only supports certain versions.
-The git hash of these versions (one for 3.x, one for 4.x) can be found by
-running `make` without a configured `ESPIDF`. Then you can fetch the
-required IDF using the following command:
-
-```bash
-$ cd ports/esp32
-$ make ESPIDF=  # This will print the supported hashes, copy the one you want.
-$ export ESPIDF=$HOME/src/github.com/espressif/esp-idf  # Or any path you like.
-$ mkdir -p $ESPIDF
-$ cd $ESPIDF
-$ git clone https://github.com/espressif/esp-idf.git $ESPIDF
-$ git checkout <Current supported ESP-IDF commit hash>
-$ git submodule update --init --recursive
-```
-
-Note: The ESP IDF v4.x support is currently experimental. It does not
-currently support PPP or wired Ethernet.
-
-Python dependencies
-===================
-
-You will also need other dependencies from the IDF, see
-`$ESPIDF/requirements.txt`, but at a minimum you need `pyserial>=3.0` and
-`pyparsing>=2.0.3,<2.4.0`.
-
-You can use Python 2 or Python 3. If you need to override the system default
-add (for example) `PYTHON=python3` to any of the `make` commands below.
-
-It is recommended to use a Python virtual environment. Even if your system
-package manager already provides these libraries, the IDF v4.x is currently
-incompatible with pyparsing 2.4 and higher.
-
-For example, to set up a Python virtual environment from scratch:
-
-```bash
-$ cd ports/esp32
-$ python3 -m venv build-venv
-$ source build-venv/bin/activate
-$ pip install --upgrade pip
-$ pip install -r path/to/esp-idf/requirements.txt
-```
+The Espressif instructions will guide you through using the `install.sh`
+(or `install.bat`) script to download the toolchain and set up your environment.
+The steps to take are summarised below.
 
-To re-enter this virtual environment in future sessions, you only need to
-source the `activate` script, i.e.:
+To check out a copy of the IDF use git clone:
 
 ```bash
-$ cd ports/esp32
-$ source build-venv/bin/activate
+$ git clone -b v4.0.2 --recursive https://github.com/espressif/esp-idf.git
 ```
 
-Then, to install the toolchain (which includes the GCC compiler, linker, binutils,
-etc), there are two options:
-
-1. Using the IDF scripts to install the toolchain (IDF 4.x only)
-================================================================
-
-Follow the steps at the [Espressif Getting Started guide](https://docs.espressif.com/projects/esp-idf/en/v4.0/get-started/index.html#step-3-set-up-the-tools).
-
-This will guide you through using the `install.sh` (or `install.bat`) script
-to download the toolchain and add it to your `PATH`. The steps are summarised
-below:
-
-After you've cloned and checked out the IDF to the correct version (see
-above), run the `install.sh` script:
-
-```bash
-$ cd $ESPIDF
-$ ./install.sh    # (or install.bat on Windows)
-```
+(You don't need a full recursive clone; see the `ci_esp32_setup` function in
+`tools/ci.sh` in this repository for more detailed set-up commands.)
 
-Then in the `ports/esp32` directory, source the `export.sh` script to set the
-`PATH`.
+After you've cloned and checked out the IDF to the correct version, run the
+`install.sh` script:
 
 ```bash
-$ cd micropython/ports/esp32
-$ source $ESPIDF/export.sh   # (or path\to\esp-idf\export.bat on Windows)
-$ # Run make etc, see below.
+$ cd esp-idf
+$ ./install.sh       # (or install.bat on Windows)
+$ source export.sh   # (or export.bat on Windows)
 ```
 
 The `install.sh` step only needs to be done once. You will need to source
 `export.sh` for every new session.
 
-Note: If you get an error about `--no-site-packages`, then modify
-`$ESPIDF/tools/idf_tools.py` and make the same change as [this
-commit](https://github.com/espressif/esp-idf/commit/7a18f02acd7005f7c56e62175a8d1968a1a9019d).
-
-2. or, Downloading pre-built toolchain manually (IDF 3.x and 4.x)
-=============================================================
-
-Note: while this works with 4.x, if you're using the 4.x IDF, it's much
-simpler to use the guide above, which will also get a more recent version of
-the toolchain.
-
-You can follow the 3.x guide at:
-
-  * [Linux installation](https://docs.espressif.com/projects/esp-idf/en/v3.3.2/get-started/linux-setup.html)
-  * [MacOS installation](https://docs.espressif.com/projects/esp-idf/en/v3.3.2/get-started/macos-setup.html)
-  * [Windows installation](https://docs.espressif.com/projects/esp-idf/en/v3.3.2/get-started/windows-setup.html)
-
-You will need to update your `PATH` environment variable to include the ESP32
-toolchain. For example, you can issue the following commands on (at least)
-Linux:
-
-    $ export PATH=$PATH:$HOME/esp/crosstool-NG/builds/xtensa-esp32-elf/bin
-
-You can put this command in your `.profile` or `.bash_login`, or do it manually.
-
-Configuring the MicroPython build
----------------------------------
-
-You then need to set the `ESPIDF` environment/makefile variable to point to
-the root of the ESP-IDF repository. The recommended way to do this is to have
-a custom `makefile` in `ports/esp32` which sets any additional variables, then
-includes the main `Makefile`. Note that GNU Make will preferentially run
-`GNUmakefile`, then `makefile`, then `Makefile`, which is what allows this to
-work. On case-insensitive filesystems, you'll need to use `GNUmakefile` rather
-than `makefile`.
-
-Create a new file in the esp32 directory called `makefile` (or `GNUmakefile`)
-and add the following lines to that file:
-
-```
-ESPIDF ?= <path to root of esp-idf repository>
-BOARD ?= GENERIC
-#PORT ?= /dev/ttyUSB0
-#FLASH_MODE ?= qio
-#FLASH_SIZE ?= 4MB
-#CROSS_COMPILE ?= xtensa-esp32-elf-
-
-include Makefile
-```
-
-Be sure to enter the correct path to your local copy of the IDF repository
-(and use `$(HOME)`, not tilde (`~`), to reference your home directory).
-
-If the Xtensa cross-compiler is not in your path you can use the
-`CROSS_COMPILE` variable to set its location.  Other options of interest are
-`PORT` for the serial port of your ESP32 module, and `FLASH_MODE` (which may
-need to be `dio` for some modules) and `FLASH_SIZE`.  See the Makefile for
-further information.
-
-The default ESP IDF configuration settings are provided by the `GENERIC`
-board definition in the directory `boards/GENERIC`. For a custom configuration
-you can define your own board directory.
-
-Any of these variables can also be set on the make command line, e.g. to set
-the `BOARD` variable, use:
-
-```bash
-$ make BOARD=TINYPICO
-```
-
-Note the use of `?=` in the `makefile` which allows them to be overridden on
-the command line. There is also a `GENERIC_SPIRAM` board for for ESP32
-modules that have external SPIRAM, but prefer to use a specific board target
-(or define your own as necessary).
-
 Building the firmware
 ---------------------
 
@@ -198,8 +73,7 @@ built-in scripts to bytecode.  This can be done by (from the root of
 this repository):
 
 ```bash
-$ cd mpy-cross
-$ make mpy-cross
+$ make -C mpy-cross
 ```
 
 Then to build MicroPython for the ESP32 run:
@@ -210,16 +84,14 @@ $ make submodules
 $ make
 ```
 
-This will produce binary firmware images in the `build/` subdirectory
-(three of them: bootloader.bin, partitions.bin and application.bin).
+This will produce a combined `firmware.bin` image in the `build-GENERIC/`
+subdirectory (this firmware image is made up of: bootloader.bin, partitions.bin
+and micropython.bin).
 
 To flash the firmware you must have your ESP32 module in the bootloader
 mode and connected to a serial port on your PC.  Refer to the documentation
-for your particular ESP32 module for how to do this.  The serial port and
-flash settings are set in the `Makefile`, and can be overridden in your
-local `makefile`; see above for more details.
-
-You will also need to have user permissions to access the /dev/ttyUSB0 device.
+for your particular ESP32 module for how to do this.
+You will also need to have user permissions to access the `/dev/ttyUSB0` device.
 On Linux, you can enable this by adding your user to the `dialout` group, and
 rebooting or logging out and in again. (Note: on some distributions this may
 be the `uucp` group, run `ls -la /dev/ttyUSB0` to check.)
@@ -242,8 +114,23 @@ To flash the MicroPython firmware to your ESP32 use:
 $ make deploy
 ```
 
-This will use the `esptool.py` script (provided by ESP-IDF) to flash the
-binary images to the device.
+The default ESP32 board build by the above commands is the `GENERIC` one, which
+should work on most ESP32 modules.  You can specify a different board by passing
+`BOARD=<board>` to the make commands, for example:
+
+```bash
+$ make BOARD=GENERIC_SPIRAM
+```
+
+Note: the above "make" commands are thin wrappers for the underlying `idf.py`
+build tool that is part of the ESP-IDF.  You can instead use `idf.py` directly,
+for example:
+
+```bash
+$ idf.py build
+$ idf.py -D MICROPY_BOARD=GENERIC_SPIRAM build
+$ idf.py flash
+```
 
 Getting a Python prompt on the device
 -------------------------------------
@@ -262,6 +149,8 @@ or
 $ miniterm.py /dev/ttyUSB0 115200
 ```
 
+You can also use `idf.py monitor`.
+
 Configuring the WiFi and using the board
 ----------------------------------------
 
@@ -301,9 +190,27 @@ import machine
 antenna = machine.Pin(16, machine.Pin.OUT, value=0)
 ```
 
+Defining a custom ESP32 board
+-----------------------------
+
+The default ESP-IDF configuration settings are provided by the `GENERIC`
+board definition in the directory `boards/GENERIC`. For a custom configuration
+you can define your own board directory.  Start a new board configuration by
+copying an existing one (like `GENERIC`) and modifying it to suit your board.
+
+MicroPython specific configuration values are defined in the board-specific
+`mpconfigboard.h` file, which is included by `mpconfigport.h`.  Additional
+settings are put in `mpconfigboard.cmake`, including a list of `sdkconfig`
+files that configure ESP-IDF settings.  Some standard `sdkconfig` files are
+provided in the `boards/` directory, like `boards/sdkconfig.ble`.  You can
+also define custom ones in your board directory.
+
+See existing board definitions for further examples of configuration.
+
+Configuration
 Troubleshooting
 ---------------
 
-* Continuous reboots after programming: Ensure FLASH_MODE is correct for your
-  board (e.g. ESP-WROOM-32 should be DIO). Then perform a `make clean`, rebuild,
-  redeploy.
+* Continuous reboots after programming: Ensure `CONFIG_ESPTOOLPY_FLASHMODE` is
+  correct for your board (e.g. ESP-WROOM-32 should be DIO). Then perform a
+  `make clean`, rebuild, redeploy.