Merge tag 'v1.8.6' into parse-bytecode

ESP8266 port uses SDK 2.0, has more heap, has support for 512k devices

This release brings some code size reductions to the core as well as
more tests and improved coverage which is now at 94.3%.

The time.ticks_diff(a, b) function has changed: the order of the arguments
has been swapped so that it behaves like "a - b", and it can now return a
negative number if "a" came before "b" (modulo the period of the ticks
functions).

For the ESP8266 port the Espressif SDK has been updated to 2.0.0, the
heap has been increased from 28k to 36k, and there is support for 512k
devices via "make 512k".  upip is included by default as frozen bytecode.
The network module now allows access-point reconnection without WiFi
credentials, and exposes configuration for the station DHCP hostname.  The
DS18B20 driver now handles negative temperatures, and NeoPixel and APA102
drivers handle 4 bytes-per-pixel LEDs.

For the CC3200 port there is now support for loading of precompiled .mpy
files and threading now works properly with interrupts.

A detailed list of changes follows.

py core:
- py.mk: automatically add frozen.c to source list if FROZEN_DIR is defined
- be more specific with MP_DECLARE_CONST_FUN_OBJ macros
- specialise builtin funcs to use separate type for fixed arg count
- {modbuiltins,obj}: use MP_PYTHON_PRINTER where possible
- modbuiltins: add builtin "slice", pointing to existing slice type
- add "delattr" builtin, conditional on MICROPY_CPYTHON_COMPAT
- sequence: fix reverse slicing of lists
- fix null pointer dereference in mpz.c, fix missing va_end in warning.c
- remove asserts that are always true in emitbc.c
- fix wrong assumption that m_renew will not move if shrinking
- change config default so m_malloc0 uses memset if GC not enabled
- add MICROPY_FLOAT_CONST macro for defining float constants
- move frozen bytecode Makefile rules from ports to common mk files
- strip leading dirs from frozen mpy files, so any path can be used

extmod:
- vfs_fat_file: check fatfs f_sync() and f_close() returns for errors
- vfs_fat_file: make file.close() a no-op if file already closed
- utime_mphal: ticks_diff(): switch arg order, return signed value
- utime_mphal: add MP_THREAD_GIL_EXIT/ENTER warppers for sleep functions
- utime_mphal: implement ticks_add(), add to all maintained ports
- utime_mphal: allow ticks functions period be configurable by a port

lib:
- utils/pyhelp.c: use mp_printf() instead of printf()
- utils/pyexec: add mp_hal_set_interrupt_char() prototype
- libm: move Thumb-specific sqrtf function to separate file

drivers:
- add "from micropython import const" when const is used

tools:
- upgrade upip to 1.1.4: fix error on unix when installing to non-existing
  absolute path
- pip-micropython: remove deprecated wrapper tool
- check_code_size.sh: code size validation script for CI
- replace upip tarball with just source file, to make its inclusion as
  frozen modules in multiple ports less magic

tests:
- extmod/vfs_fat: improve VFS test coverage
- basics/builtin_slice: add test for "slice" builtin name
- basics: add test for builtin "delattr"
- extmod/vfs_fat_fsusermount: improve fsusermount test coverage
- extmod/vfs_fat_oldproto: test old block device protocol
- basics/gc1: garbage collector threshold() coverage
- extmod/uhashlib_sha1: coverage for SHA1 algorithm
- extmod/uhashlib_sha256: rename sha256.py test
- btree1: fix out of memory error running on esp8266
- extmod/ticks_diff: test for new semantics of ticks_diff()
- extmod/framebuf1: test framebuffer pixel clear, and text function

minimal port:
- Makefile: split rule for firmware.bin generation

unix port:
- Makefile: remove references to deprecated pip-micropython
- modtime: use ticks_diff() implementation from extmod/utime_mphal.c
- mphalport.h: add warning of mp_hal_delay_ms() implementation
- modtime: switch ticks/sleep_ms/us() to utime_mphal
- fix symbol references for x86 Mac
- replace upip tarball with just source file

windows port:
- enable utime_mphal following unix, define mp_hal_ticks_*
- fix utime_mphal compilation for msvc
- implement mp_hal_ticks_cpu in terms of QueryPerformanceCounter

qemu-arm port:
- exclude ticks_diff test for qemu-arm port
- exclude extmod/vfs_fat_fileio.py test
- exclude new vfs_fat tests
- enable software floating point support, and float tests

stmhal port:
- modutime: refactor to use extmod's version of ticks_cpu
- refactor pin usage to use mp_hal_pin API
- led: refactor LED to use mp_hal_pin_output() init function
- Makefile: use standard rules for frozen module generation
- modutime: consistently convert to MP_ROM_QSTR/MP_ROM_PTR
- enable SD power save (disable CLK on idle)

cc3200 port:
- use mp_raise_XXX helper functions to reduce code size
- mods/pybspi: allow "write" arg of read/readinto to be positional
- enable loading of precompiled .mpy files
- fix thread mutex's so threading works with interrupts

teensy port:
- update to provide new mp_hal_pin_XXX functions following stmhal

esp8266 port:
- Makefile: use latest esptool.py flash size auto-detection
- esp_init_data: auto-initialize system params with vendor SDK 2.0.0
- esp8266.ld: move help.o to iROM
- esp8266.ld: move modmachine.o to iROM
- esp8266.ld: move main.o to iROM
- add MP_FASTCODE modifier to put a function to iRAM
- main: mark nlr_jump_fail() as MP_FASTCODE
- modules/webrepl: enforce only one concurrent WebREPL connection
- etshal.h: add few more ESP8266 vendor lib prototypes
- modesp: add flash_user_start() function
- add support for building firmware version for 512K modules
- scripts: make neopixel/apa102 handle 4bpp LEDs with common code
- modutime: consistently convert to MP_ROM_QSTR/MP_ROM_PTR
- modnetwork: config(): fix copy-paste error in setting "mac"
- scripts/port_diag: add descriptions for esf_buf types
- modnetwork.c: allows AP reconnection without WiFi credentials
- main: bump heap size to 36K
- etshal.h: add prototypes for SPIRead/SPIWrite/SPIEraseSector
- etshal.h: adjust size of MD5_CTX structure
- modules: fix negative temperature in ds18x20 driver
- rename "machine" module implementation to use contemporary naming
- rework webrepl_setup to run over wired REPL
- espneopixel.c: solve glitching LED issues with cpu at 80MHz
- include upip as a standard frozen bytecode module
- update docs for esptool 1.2.1/SDK 2.0 (--flash_size=detect)
- modnetwork.c: expose configuration for station DHCP hostname

zephyr port:
- implement utime module
- use board/SoC values for startup banner based on Zephyr config
- initial implementation of machine.Pin
- zephyr_getchar: update for recent Zephyr refactor of console hooks
- support time -> utime module "weaklink"
- README: update for the current featureset, add more info
- mpconfigport.h: move less important params to the bottom
- Makefile: allow to adjust heap size from make command line
- Makefile: update comments to the current state of affairs
- Makefile: allow to override Zephyr config from make command line
- Makefile: add minimal port
- Makefile: add -fomit-frame-pointer to reduce code size
- mphalport.h: update for new "unified" kernal API (sleep functions)

docs:
- machine.SPI: bring up to date with Hardware API, make vendor-neutral
- machine.SPI: improve descriptions of xfer methods
- library/builtins: add docs for delattr and slice
- library/network: reword intro paragraph
- library/network: typo fixes, consistent acronym capitalization
- library/index: update TOCs so builtins sorted before modules
- utime: document ticks_cpu() in more detail
- utime: describe new semantics of ticks_diff() (signed ring arithmetics)
- utime: add docs for ticks_add(), improvements for other ticks_*()
- esp8266: update for new WebREPL setup procedure
- */quickref.rst: use new semantics of ticks_diff()
- library/machine.Pin: update Pin docs to align with new HW API

travis:
- integrate tools/check_code_size.sh
- minimal: Use CROSS=1, for binary size check

examples:
- http_server_simplistic: add "not suitable for real use" note
- hwapi: example showing best practices for HW API usage in apps
- hwapi: add hwconfig for DragonBoard 410c

5 files changed, 186 insertions, 0 deletions

diff --git a/examples/hwapi/README.md b/examples/hwapi/README.md
new file mode 100644
index 000000000..1992eb660
--- /dev/null
+++ b/examples/hwapi/README.md
@@ -0,0 +1,126 @@
+This directory shows the best practices for using MicroPython hardware API
+(`machine` module). `machine` module strives to provide consistent API
+across various boards, with the aim to enable writing portable applications,
+which would work from a board to board, from a system to another systems.
+This is inherently a hard problem, because hardware is different from one
+board type to another, and even from examplar of board to another. For
+example, if your app requires an external LED, one user may connect it
+to one GPIO pin, while another user may find it much more convinient to
+use another pin. This of course applies to relays, buzzers, sensors, etc.
+
+With complications above in mind, it's still possible to write portable
+applications by using "low[est] denominator" subset of hardware API and
+following simple rules outlined below. The applications won't be able
+to rely on advanced hardware capabilities of a particular board and
+will be limited to generic capabilities, but it's still possible to
+write many useful applications in such a way, with the obvious benefit of
+"write once - run everywhere" approach (only configuration for a particular
+board is required).
+
+The key to this approach is splitting your application into (at least)
+2 parts:
+
+* main application logic
+* hardware configuration
+
+The key point is that hardware configuration should be a separate file
+(module in Python terms). A good name would be `hwconfig.py`, and that's
+how we'll call it from now on. Another key point is that main application
+should never instantiate (construct) hardware objects directly. Instead,
+they should be defined in `hwconfig.py`, and main application should
+import and reference hardware objects via this module. The simplest
+application of this idea would look like:
+
+`hwconfig.py`:
+
+    from machine import Pin
+
+    LED = Pin("A3", Pin.OUT)
+
+`app.py`:
+
+    from hwconfig import *
+    import utime
+
+    while True:
+        LED.value(1)
+        utime.sleep_ms(500)
+        LED.value(0)
+        utime.sleep_ms(500)
+
+
+To deploy this application to a particular board, a user will need:
+
+1. Edit `hwconfig.py` to adjust Pin and other hardware peripheral
+   parameters and locations.
+2. Actually deploy `hwconfig.py` and `app.py` to a board (e.g. copy to
+   board's filesystem, or build new firmware with these modules frozen
+   into it).
+
+Note that there's no need to edit the main application code! (Which may
+be complex, while `hwconfig.py` should usually remain short enough, and
+focused solely on hardware configuration).
+
+An obvious improvement to this approach is the following. There're few
+well-known boards which run MicroPython, and most of them include an
+onboard LED. So, to help users of these boards to do configuration
+quickly (that's especially important for novice users, for who may
+be stumped by the need to reach out to a board reference to find LED
+pin assignments), `hwconfig.py` your application ships may include
+commented out sections with working configurations for different
+boards. The step 1 above then will be:
+
+1. Look thru `hwconfig.py` to find a section which either exactly
+   matches your board, or the closest to it. Uncomment, and if any
+   adjustments required, apply them.
+
+It's important to keep in mind that adjustments may be always required,
+and that there may be users whose configuration doesn't match any of
+the available. So, always include a section or instructions for them.
+Consider for example that even on a supported board, user may want to
+blink not an on-board LED, but the one they connected externally.
+MicroPython's Hardware API offers portability not just among "supported"
+boards, but to any board at all, so make sure users can enjoy it.
+
+There's next step of improvement to make. While having one `hwconfig.py`
+with many sections would work for smaller projects with few hardware
+objects, it may become more cumbersome to maintain both on programmer's
+and user's sides for larger projects. Then instead of single
+`hwconfig.py` file, you can provide few "template" ones for well-known
+boards:
+
+* `hwconfig_pyboard.py`
+* `hwconfig_wipy.py`
+* `hwconfig_esp8266.py`
+* etc.
+
+Then step 1 above will be:
+
+1. Look thru available `hwconfig_*.py` files and find one which matches
+   your board the best, then rename to `hwconfig.py` and make adjustments,
+   if any.
+
+Again, please keep in mind that there may be users whose hardware will be
+completely unlike you heard of. Give them some helpful hints too, perhaps
+provide `hwconfig_custom.py` with some instructions.
+
+That's where we stop with improvements to the "separate file for hardware
+configuration" idea, as it is already pretty flexible and viable. An
+application in this directory shows it in practice, using slightly less
+trivial example than just a blinking LED: `soft_pwm.py` implements a
+software PWM (pulse width modulation) to produce an LED fade-in/fade-out
+effect - without any dependence on hardware PWM availability.
+
+Note that improvements to board configuration handling may continue further.
+For example, one may invent a "configuration manager" helper module which will
+try to detect current board (among well-known ones), and load appropriate
+`hwconfig_*.py` - this assumes that a user would lazily deploy them all
+(or that application will be automatically installed, e.g. using MicroPython's
+`upip` package manager). The key point in this case remains the same as
+elaborated above - always assume there can, and will be a custom configuration,
+and it should be well supported. So, any automatic detection should be
+overridable by a user, and instructions how to do so are among the most
+important you may provide for your application.
+
+By following these best practices, you will use MicroPython at its full
+potential, and let users enjoy it too. Good luck!
diff --git a/examples/hwapi/hwconfig_dragonboard410c.py b/examples/hwapi/hwconfig_dragonboard410c.py
new file mode 100644
index 000000000..00f21a658
--- /dev/null
+++ b/examples/hwapi/hwconfig_dragonboard410c.py
@@ -0,0 +1,12 @@
+from machine import Pin
+
+# 96Boards/Qualcomm DragonBoard 410c
+# By default, on-board LEDs are controlled by kernel LED driver.
+# To make corresponding pins be available as normal GPIO,
+# corresponding driver needs to be unbound first (as root):
+# echo -n "soc:leds" >/sys/class/leds/apq8016-sbc:green:user1/device/driver/unbind
+# Note that application also either should be run as root, or
+# /sys/class/gpio ownership needs to be changed.
+
+# User LED 1 on gpio21
+LED = Pin(21, Pin.OUT)
diff --git a/examples/hwapi/hwconfig_esp8266_esp12.py b/examples/hwapi/hwconfig_esp8266_esp12.py
new file mode 100644
index 000000000..e8cf2d12e
--- /dev/null
+++ b/examples/hwapi/hwconfig_esp8266_esp12.py
@@ -0,0 +1,5 @@
+from machine import Pin
+
+# ESP12 module as used by many boards
+# Blue LED on pin 2
+LED = Pin(2, Pin.OUT)
diff --git a/examples/hwapi/hwconfig_z_frdm_k64f.py b/examples/hwapi/hwconfig_z_frdm_k64f.py
new file mode 100644
index 000000000..c45e3e756
--- /dev/null
+++ b/examples/hwapi/hwconfig_z_frdm_k64f.py
@@ -0,0 +1,5 @@
+from machine import Pin
+
+# Freescale/NXP FRDM-K64F board
+# Blue LED on port B, pin 21
+LED = Pin(("GPIO_1", 21), Pin.OUT)
diff --git a/examples/hwapi/soft_pwm.py b/examples/hwapi/soft_pwm.py
new file mode 100644
index 000000000..a9a556171
--- /dev/null
+++ b/examples/hwapi/soft_pwm.py
@@ -0,0 +1,38 @@
+import utime
+from hwconfig import LED
+
+
+# Using sleep_ms() gives pretty poor PWM resolution and
+# brightness control, but we use it in the attempt to
+# make this demo portable to even more boards (e.g. to
+# those which don't provide sleep_us(), or provide, but
+# it's not precise, like would be on non realtime OSes).
+# We otherwise use 20ms period, to make frequency not less
+# than 50Hz to avoid visible flickering (you may still see
+# if you're unlucky).
+def pwm_cycle(led, duty, cycles):
+    duty_off = 20 - duty
+    for i in range(cycles):
+        if duty:
+            led.value(1)
+            utime.sleep_ms(duty)
+        if duty_off:
+            led.value(0)
+            utime.sleep_ms(duty_off)
+
+
+# At the duty setting of 1, an LED is still pretty bright, then
+# at duty 0, it's off. This makes rather unsmooth transition, and
+# breaks fade effect. So, we avoid value of 0 and oscillate between
+# 1 and 20. Actually, highest values like 19 and 20 are also
+# barely distinguishible (like, both of them too bright and burn
+# your eye). So, improvement to the visible effect would be to use
+# more steps (at least 10x), and then higher frequency, and use
+# range which includes 1 but excludes values at the top.
+while True:
+    # Fade in
+    for i in range(1, 21):
+        pwm_cycle(LED, i, 2)
+    # Fade out
+    for i in range(20, 0, -1):
+        pwm_cycle(LED, i, 2)