Merge tag 'v1.8.4' into parse-bytecode

Support for stream decompression in uzlib, and more ESP8266 features

This release includes some bug fixes, code clean-up, updates to the docs,
more tests, and various feature additions.  The uzlib module now supports
efficient stream decompression in the form of the uzlib.DecompIO class.
Freezing of bytecode now supports floats for the ESP8266 port, as well as
complex numbers for all ports.  The stmhal port has ADC working on L4
microcontrollers, fixed initialisation for DAC, and addition of the
machine.WDT class and machine.reset_cause function.

For the ESP8266 port Pin(16) now works as an input pin and the hardware
SPI peripheral is exposed as machine.SPI(1).  The os.umount function is
implemented and the port supports mounting of externally connected SD
cards.  The machine.WDT class is added, wlan.scan() is fixed to return all
access points, and there is support for DS18S20 devices.

py core:
- runtime: factor out exception raising helpers
- runtime: define mp_check_self(pred) helper macro
- objdict: get rid of asserts (remove/replace with mp_check_self())
- get rid of assert() in method argument checking functions
- objtuple: in tuple_cmp_helper, use mp_check_self instead of raising
- objstr: use mp_raise_{Type,Value}Error instead of mp_raise_msg
- obj.h: for obj reprs A,B,C use void* explicitly for mp_obj_t typedef
- mpconfigport.h: remove typedef of machine_ptr_t, it's no longer needed
- sequence: allow to use bignums as indices in slice objects
- stream.c: use mp_obj_get_type in mp_get_stream_raise
- gc: add MICROPY_GC_CONSERVATIVE_CLEAR option to always zero memory
- compile: don't compile assert statements when optimisations enabled
- modstruct: use more compact mp_raise_ValueError function
- emitglue: use more compact mp_raise_ValueError function
- rename struct mp_code_state to mp_code_state_t
- mkrules.mk: allow to override name of libmicropython.a
- mpprint: fail an assertion with unsupported format specifiers
- makeqstrdata.py: compute the qstr hash from bytes, not characters
- if str/bytes hash is 0 then explicitly compute it
- emitglue.c: provide mp_raw_code_load_file for any unix architecture
- add MICROPY_USE_INTERNAL_PRINTF option, defaults to enabled

extmod:
- modwebrepl: set_password(): raise exception for too long password
- uzlib/: update uzlib to v2.0: new API supporting stream decompression
- moduzlib: refactor to new stream-compatible uzlib 2.0 API
- uzlib/: update uzlib to v2.0.1: fixes for pedantic compiler warnings
- uzlib/: update uzlib to v2.0.2: consistently use stdint types
- modbtree: do CHECK_ERROR after __bt_seq()
- modubinascii: implement binascii.crc32
- modubinascii: make crc32() support configurable
- modframebuf: fix pixel accessor to return a 1-bit result
- add machine_spi with generic SPI C-protocol and helper methods
- modframebuf: fix fill and scroll when height not divisible by 8
- moduzlib: implement zlib stream decompressor class, DecompIO
- moduzlib: use mperrno.h for error constants
- modframebuf: include font from stmhal directory explicitly
- moduzlib: support wbits arg to DecompIO
- framebuf: add the xstep!=0 case to scroll() method

lib:
- utils/stdout_helpers: fix function signature to match py/mphal.h
- berkeley-db-1.xx: update to upstream, fixes MacOSX build
- utils/pyexec: qstr_pool_info() requires size_t* parameters

drivers:
- sdcard: port the SDCard driver to new machine API, with backwards
  compatibility for pyboard

tools:
- mpy-tool.py: support freezing float literals with obj-repr C
- mpy-tool.py: store qstr config values in global config object
- mpy-tool.py: compute the hash value for str/bytes objects
- mpy-tool.py: support freezing of complex numbers

tests:
- rename zlibd_decompress.py -> uzlib_decompress.py
- basics: add more tuple tests to improve coverage testing
- basics: add more list tests to improve coverage testing
- misc/non_compliant: add tests to improve coverage testing
- basics: add test for break from within try within a for-loop
- basics: add a test file for overriding special methods
- basics/special_methods: enable tests for extra special methods
- uzlib_decompress: actually test raw DEFLATE stream
- run-tests: disable thread/thread_lock4.py on Travis
- run-tests: disable thread/stress_heap.py when running on Travis
- cmdline: add test for -O option to check optimisation value
- extmod/vfs_fat_ramdisk: add tests for VFS.umount()
- run-tests: disable thread_gc1.py test on Travis
- unix/extra_coverage: add test for str/bytes with invalid hash
- extmod: add test for uzlib.DecompIO
- extmod: add a test for framebuf module, tested by coverage build
- extmod/uzlib_decompio: add zlib bitstream testcases
- extmod/framebuf1: add tests for scrolling in the x-direction
- run-tests: disable thread/stress_recurse.py test on Travis

unix port:
- mpconfigport.h: don't include stdio.h on MacOS
- when find'ing frozen files don't use extra slash, do follow symlinks

qemu-arm port:
- enable MICROPY_PY_ALL_SPECIAL_METHODS

stmhal port:
- boards: update STM32L476 pin defs to include ADC channels
- adc.c: get ADC working on STM32L4 MCUs
- fix timer capture/compare interrupt handling for TIM1 and TIM8
- remove obsolete code for special handling of TIM3 irq settings
- make ADC channel 16 available on L4 MCUs
- update pin print to print new constants
- modusocket: set self->nic to MP_OBJ_NULL after socket close
- update boot.py files to use VCP instead of CDC
- spi: factor out SPI transfer code to a single function
- spi: support new machine SPI methods in legacy SPI object
- add machine.WDT class
- set STM32F7DISC CPU Frequency to 216 MHz
- dac: fix DAC (re-)initialisation by resetting DMA
- wdt: implement keyword args to WDT constructor
- modmachine: implement machine.reset_cause() function, and consts
- machine.POWER_ON is renamed to machine.PWRON_RESET
- when find'ing frozen files don't use extra slash, do follow symlinks

cc3200 port:
- add machine.PWRON_RESET constant (machine.POWER_ON is now deprecated)

teensy port:
- fix execution of frozen boot.py and main.py

esp8266 port:
- fix reading of pin object for GPIO16; Pin(16) now works as an input
- PULL_UP is not supported on Pin(16), so raise an exception in this case
- enable support for all special methods
- modpybhspi: add a HSPI module for hardware SPI support
- modmachinespi: add a factory method for SoftSPI/HSPI
- esp_mphal: no longer disable watchdog on startup
- modpybrtc: use 64-bit arithmetic when computing alarm expiry
- hspi: enable duplex operation of hardware SPI
- modous: add os.umount method to unmount a filesystem
- modmachinewdt: implement machine.WDT class
- modules: split onewire.py into OneWire and DS18X20 driver
- modules/onewire: change onewire.read() to onewire.readinto()
- modules/ds18x20.py: add support for DS18S20 devices
- modpybspi: use generic SPI helper methods to implement SPI
- modpybhspi: simplify HSPI driver by using 1 function for xfers
- modmachinewdt: add .deinit() method
- modmachine: add WDT_RESET and SOFT_RESET constants
- modmachine: don't expose internal SoftSPI and HSPI classes
- modmachine: simplify SPI class implementation multiplexing
- espneopixel: disable IRQs during eps.neopixel_write
- modnetwork: fix wlan.scan() method so it returns all networks
- modmachine: map PWR_ON_RESET to vendor's REASON_DEFAULT_RST
- machine.PWR_ON_RESET is renamed to machine.PWRON_RESET
- when find'ing frozen files don't use extra slash, do follow symlinks

docs:
- esp8266/tutorial/pins: fix typo in commands for pin input mode
- esp8266/intro: add command to install esptool.py 1.0.1 via pip
- library/machine.WDT: add note that WDT is only available on WiPy
- esp8266/quickref: fix and update the SPI docs
- esp8266: update quickref and tutorial for OneWire/DS18X20 driver
- pyboard: update USB mouse tutorial to use VCP instead of CDC
- pyboard: update USB mouse tutorial to use pyb.USB_HID()
- library: add reference for pyb.usb_mode and pyb.USB_HID
- pyboard/quickref: add links to pinouts for other pyboard variants
- pyboard/quickref: add section on "delay and timing" for utime mod
- esp8266/quickref: add internal links to docs for some modules
- esp8266/quickref: update information on SPI classes
- esp8266/quickref: further improvements for SPI subsections
- library/machine.WDT: add that WDT is available on pyboard
- reference/isr_rules.rst: two minor additions to docs for using ISR

misc:
- add *.pyc to .gitignore, because Python 2 doesn't use __pycache__
- build mpy-cross as part of the Travis process

1 files changed, 224 insertions, 0 deletions

diff --git a/esp8266/modpybhspi.c b/esp8266/modpybhspi.c
new file mode 100644
index 000000000..c1cd7f662
--- /dev/null
+++ b/esp8266/modpybhspi.c
@@ -0,0 +1,224 @@
+/*
+ * This file is part of the MicroPython project, http://micropython.org/
+ *
+ * The MIT License (MIT)
+ *
+ * Copyright (c) 2016 Damien P. George
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include <stdio.h>
+#include <stdint.h>
+#include <string.h>
+
+#include "ets_sys.h"
+#include "etshal.h"
+#include "ets_alt_task.h"
+
+#include "py/runtime.h"
+#include "py/stream.h"
+#include "py/mphal.h"
+#include "extmod/machine_spi.h"
+
+#include "hspi.h"
+
+mp_obj_t pyb_spi_make_new(const mp_obj_type_t *type, size_t n_args,
+                          size_t n_kw, const mp_obj_t *args);
+
+typedef struct _pyb_hspi_obj_t {
+    mp_obj_base_t base;
+    uint32_t baudrate;
+    uint8_t polarity;
+    uint8_t phase;
+} pyb_hspi_obj_t;
+
+
+STATIC void hspi_transfer(mp_obj_base_t *self_in, size_t src_len, const uint8_t *src_buf, size_t dest_len, uint8_t *dest_buf) {
+    (void)self_in;
+
+    if (dest_len == 0) {
+        // fast case when we only need to write data
+        size_t chunk_size = 1024;
+        size_t count = src_len / chunk_size;
+        size_t i = 0;
+        for (size_t j = 0; j < count; ++j) {
+            for (size_t k = 0; k < chunk_size; ++k) {
+                spi_tx8fast(HSPI, src_buf[i]);
+                ++i;
+            }
+            ets_loop_iter();
+        }
+        while (i < src_len) {
+            spi_tx8fast(HSPI, src_buf[i]);
+            ++i;
+        }
+    } else {
+        // we need to read and write data
+
+        // Process data in chunks, let the pending tasks run in between
+        size_t chunk_size = 1024; // TODO this should depend on baudrate
+        size_t count = dest_len / chunk_size;
+        size_t i = 0;
+        for (size_t j = 0; j < count; ++j) {
+            for (size_t k = 0; k < chunk_size; ++k) {
+                uint32_t data_out;
+                if (src_len == 1) {
+                    data_out = src_buf[0];
+                } else {
+                    data_out = src_buf[i];
+                }
+                dest_buf[i] = spi_transaction(HSPI, 0, 0, 0, 0, 8, data_out, 8, 0);
+                ++i;
+            }
+            ets_loop_iter();
+        }
+        while (i < dest_len) {
+            uint32_t data_out;
+            if (src_len == 1) {
+                data_out = src_buf[0];
+            } else {
+                data_out = src_buf[i];
+            }
+            dest_buf[i] = spi_transaction(HSPI, 0, 0, 0, 0, 8, data_out, 8, 0);
+            ++i;
+        }
+    }
+}
+
+/******************************************************************************/
+// MicroPython bindings for HSPI
+
+STATIC void pyb_hspi_print(const mp_print_t *print, mp_obj_t self_in, mp_print_kind_t kind) {
+    pyb_hspi_obj_t *self = MP_OBJ_TO_PTR(self_in);
+    mp_printf(print, "HSPI(id=1, baudrate=%u, polarity=%u, phase=%u)",
+        self->baudrate, self->polarity, self->phase);
+}
+
+STATIC void pyb_hspi_init_helper(pyb_hspi_obj_t *self, size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) {
+    enum { ARG_id, ARG_baudrate, ARG_polarity, ARG_phase };
+    static const mp_arg_t allowed_args[] = {
+        { MP_QSTR_id, MP_ARG_INT, {.u_int = -1} },
+        { MP_QSTR_baudrate, MP_ARG_INT, {.u_int = -1} },
+        { MP_QSTR_polarity, MP_ARG_INT, {.u_int = -1} },
+        { MP_QSTR_phase, MP_ARG_INT, {.u_int = -1} },
+    };
+    mp_arg_val_t args[MP_ARRAY_SIZE(allowed_args)];
+    mp_arg_parse_all(n_args, pos_args, kw_args, MP_ARRAY_SIZE(allowed_args),
+                     allowed_args, args);
+
+    if (args[ARG_baudrate].u_int != -1) {
+        self->baudrate = args[ARG_baudrate].u_int;
+    }
+    if (args[ARG_polarity].u_int != -1) {
+        self->polarity = args[ARG_polarity].u_int;
+    }
+    if (args[ARG_phase].u_int != -1) {
+        self->phase = args[ARG_phase].u_int;
+    }
+    if (self->baudrate == 80000000L) {
+        // Special case for full speed.
+        spi_init_gpio(HSPI, SPI_CLK_80MHZ_NODIV);
+        spi_clock(HSPI, 0, 0);
+    } else if (self->baudrate > 40000000L) {
+        nlr_raise(mp_obj_new_exception_msg(&mp_type_ValueError,
+            "impossible baudrate"));
+    } else {
+        uint32_t divider = 40000000L / self->baudrate;
+        uint16_t prediv = MIN(divider, SPI_CLKDIV_PRE + 1);
+        uint16_t cntdiv = (divider / prediv) * 2; // cntdiv has to be even
+        if (cntdiv > SPI_CLKCNT_N + 1 || cntdiv == 0 || prediv == 0) {
+            nlr_raise(mp_obj_new_exception_msg(&mp_type_ValueError,
+                "impossible baudrate"));
+        }
+        self->baudrate = 80000000L / (prediv * cntdiv);
+        spi_init_gpio(HSPI, SPI_CLK_USE_DIV);
+        spi_clock(HSPI, prediv, cntdiv);
+    }
+    // TODO: Make the byte order configurable too (discuss param names)
+    spi_tx_byte_order(HSPI, SPI_BYTE_ORDER_HIGH_TO_LOW);
+    spi_rx_byte_order(HSPI, SPI_BYTE_ORDER_HIGH_TO_LOW);
+    CLEAR_PERI_REG_MASK(SPI_USER(HSPI), SPI_FLASH_MODE | SPI_USR_MISO |
+                        SPI_USR_ADDR | SPI_USR_COMMAND | SPI_USR_DUMMY);
+    // Clear Dual or Quad lines transmission mode
+    CLEAR_PERI_REG_MASK(SPI_CTRL(HSPI), SPI_QIO_MODE | SPI_DIO_MODE |
+                        SPI_DOUT_MODE | SPI_QOUT_MODE);
+    spi_mode(HSPI, self->phase, self->polarity);
+}
+
+mp_obj_t pyb_hspi_make_new(const mp_obj_type_t *type, size_t n_args, size_t n_kw, const mp_obj_t *args) {
+    mp_arg_check_num(n_args, n_kw, 0, 1, true);
+    mp_int_t id = -1;
+    if (n_args > 0) {
+        id = mp_obj_get_int(args[0]);
+    }
+
+    if (id == -1) {
+        // Multiplex to bitbanging SPI
+        if (n_args > 0) {
+            args++;
+        }
+        return pyb_spi_make_new(type, 0, n_kw, args);
+    }
+
+    if (id != 1) {
+        // FlashROM is on SPI0, so far we don't support its usage
+        mp_raise_ValueError("");
+    }
+
+    pyb_hspi_obj_t *self = m_new_obj(pyb_hspi_obj_t);
+    self->base.type = &pyb_hspi_type;
+    // set defaults
+    self->baudrate = 80000000L;
+    self->polarity = 0;
+    self->phase = 0;
+    mp_map_t kw_args;
+    mp_map_init_fixed_table(&kw_args, n_kw, args + n_args);
+    pyb_hspi_init_helper(self, n_args, args, &kw_args);
+    return MP_OBJ_FROM_PTR(self);
+}
+
+STATIC mp_obj_t pyb_hspi_init(size_t n_args, const mp_obj_t *args, mp_map_t *kw_args) {
+    pyb_hspi_init_helper(args[0], n_args - 1, args + 1, kw_args);
+    return mp_const_none;
+}
+MP_DEFINE_CONST_FUN_OBJ_KW(pyb_hspi_init_obj, 1, pyb_hspi_init);
+
+STATIC const mp_rom_map_elem_t pyb_hspi_locals_dict_table[] = {
+    { MP_ROM_QSTR(MP_QSTR_init), MP_ROM_PTR(&pyb_hspi_init_obj) },
+    { MP_ROM_QSTR(MP_QSTR_read), MP_ROM_PTR(&mp_machine_spi_read_obj) },
+    { MP_ROM_QSTR(MP_QSTR_readinto), MP_ROM_PTR(&mp_machine_spi_readinto_obj) },
+    { MP_ROM_QSTR(MP_QSTR_write), MP_ROM_PTR(&mp_machine_spi_write_obj) },
+    { MP_ROM_QSTR(MP_QSTR_write_readinto), MP_ROM_PTR(&mp_machine_spi_write_readinto_obj) },
+};
+
+STATIC MP_DEFINE_CONST_DICT(pyb_hspi_locals_dict, pyb_hspi_locals_dict_table);
+
+STATIC const mp_machine_spi_p_t pyb_hspi_p = {
+    .transfer = hspi_transfer,
+};
+
+const mp_obj_type_t pyb_hspi_type = {
+    { &mp_type_type },
+    .name = MP_QSTR_HSPI,
+    .print = pyb_hspi_print,
+    .make_new = pyb_hspi_make_new,
+    .protocol = &pyb_hspi_p,
+    .locals_dict = (mp_obj_dict_t*)&pyb_hspi_locals_dict,
+};