aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-08-27tests/run-tests: Use absolute paths where possible.stijn
Replace some usages of paths relative to the current working directory with absolute paths relative to the tests directory. Fixes and resulting changes: - default values of MICROPYTHON and MPYCROSS are absolute paths and always correct - likewise, the correct full paths for tools and extmod directories are appended to sys.path - printing/cleaning failures works properly since it expects the .exp and .out files in the tests directory which is also where they are written to now, plus no more need for changing directories This fixes #5872 and allows running custom tests which use run-tests without having to cd to the tests directory first, and the test output still is in the tests/ directory instead of the current working directory. Discovery of tests and all skip test logic based on paths relative to the current working directory remains unchanged which essentially means that for running most of MicroPython's own tests, run-tests must still be ran from within it's directory, so document that.
2020-08-26nrf/Makefile: Improve user C modules support.Roberto Colistete Jr
Add CFLAGS_EXTRA to CFLAGS. Include LDFLAGS_MOD to the compilation. And, add SRC_MOD to SRC_QSTR.
2020-08-26tests/extmod: Make uasyncio_fair test more reliable by adjusting sleeps.Damien George
With sleep(0.2) a multiple of sleep(0.1), the order of task 2 and 3 execution is not well defined, and depends on the precision of the system clock and how fast the rest of the code runs. So change 0.2 to 0.18 to make the test more reliable. Also fix a typo of t3/t4, and cancel t4 at the end. Signed-off-by: Damien George <damien@micropython.org>
2020-08-26extmod/bluetooth: Support active scanning in BLE.gap_scan().Andrew Leech
This adds an additional optional parameter to gap_scan() to select active scanning, where scan responses are returned as well as normal scan results. This parameter is False by default which retains the existing behaviour.
2020-08-26extmod/modbluetooth: Fix race between READ_REQUEST and other IRQs.Jim Mussared
The READ_REQUEST callback is handled as a hard interrupt (because the BLE stack needs an immediate response from it so it can continue) and so calls to Python require extra protection: - the caller-owned tuple passed into the callback must be separate from the tuple used by other callback events (which are soft interrupts); - the GC and scheduler must be locked during callback execution.
2020-08-26extmod/btstack: Implement GAP scan duration_ms parameter.Jim Mussared
This commit makes scanning work when duration_ms is set to zero. Prior to this it would not work with duration_ms set to zero.
2020-08-25extmod/vfs_lfs: Add mtime support to littlefs files.Damien George
This commit adds support for modification time of files on littlefs v2 filesystems, using file attributes. For some background see issue #6114. Features/properties of this implementation: - Only supported on littlefs2 (not littlefs1). - Uses littlefs2's general file attributes to store the timestamp. - The timestamp is 64-bits and stores nanoseconds since 1970/1/1 (if the range to the year 2554 is not enough then additional bits can be added to this timestamp by adding another file attribute). - mtime is enabled by default but can be disabled in the constructor, eg: uos.mount(uos.VfsLfs2(bdev, mtime=False), '/flash') - It's fully backwards compatible, existing littlefs2 filesystems will work without reformatting and timestamps will be added transparently to existing files (once they are opened for writing). - Files without timestamps will open correctly, and stat will just return 0 for their timestamp. - mtime can be disabled or enabled each mount time and timestamps will only be updated if mtime is enabled (otherwise they will be untouched). Signed-off-by: Damien George <damien@micropython.org>
2020-08-22py/mphal.h: Introduce mp_hal_time_ns and implement on various ports.Damien George
This should return a 64-bit value being the number of nanoseconds since 1970/1/1. Signed-off-by: Damien George <damien@micropython.org>
2020-08-22lib/timeutils: Add helper functions to deal with nanosecs since 1970.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2020-08-22unix/fatfs_port: Implement get_fattime.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2020-08-22extmod/uasyncio: Truncate negative sleeps to 0.Damien George
Otherwise a task that continuously awaits on a large negative sleep can monopolise the scheduler (because its wake time is always less than everything else in the pairing heap). Signed-off-by: Damien George <damien@micropython.org>
2020-08-22tests/extmod: Add test for uasyncio.sleep of a negative time.Damien George
It should take 0 time to await on a negative sleep. Signed-off-by: Damien George <damien@micropython.org>
2020-08-22py/runtime: Fix builtin compile() in "single" mode so it prints exprs.Damien George
As per CPython behaviour, compile(stmt, "file", "single") should create code which prints to stdout (via __repl_print__) the results of any expressions in stmt. Signed-off-by: Damien George <damien@micropython.org>
2020-08-22tools/makemanifest.py: Use os.makedirs to make path for generated files.Damien George
The existing implementation of mkdir() in this file is not sophisticated enough to work correctly on all operating systems (eg Mac can raise EISDIR). Using the standard os.makedirs() function handles all cases correctly. Signed-off-by: Damien George <damien@micropython.org>
2020-08-21tools/mpy-tool.py: Fix offset of line number info.Martin Milata
Signed-off-by: Martin Milata <martin@martinmilata.cz>
2020-08-21tools/pyboard.py: Replace eval() of received data with alternative.Michael Buesch
Prior to this commit, pyboard.py used eval() to "parse" file data received from the board. Using eval() on received data from a device is dangerous, because a malicious device may inject arbitrary code execution on the PC that is doing the operation. Consider the following scenario: Eve may write a malicious script to Bob's board in his absence. On return Bob notices that something is wrong with the board, because it doesn't work as expected anymore. He wants to read out boot.py (or any other file) to see what is wrong. What he gets is a remote code execution on his PC. Proof of concept: Eve: $ cat boot.py _print = print print = lambda *x, **y: _print("os.system('ls /; echo Pwned!')", end="\r\n\x04") $ ./pyboard.py -f cp boot.py : cp boot.py :boot.py Bob: $ ./pyboard.py -f cp :boot.py /tmp/foo cp :boot.py /tmp/foo bin chroot dev home lib32 media opt root sbin sys usr boot config etc lib lib64 mnt proc run srv tmp var Pwned! There's also the possibility that the device is malfunctioning and sends random and possibly dangerous data back to the PC, to be eval'd. Fix this problem by using ast.literal_eval() to parse the received bytes, instead of eval(). Signed-off-by: Michael Buesch <m@bues.ch>
2020-08-21stm32/pin_defs_stm32: Fix pin printing to show IN mode correctly.Dave Hylands
Prior to this commit, if you configure a pin as an output type (I2C in this example) and then later configure it back as an input, then it will report the type incorrectly. Example: >>> import machine >>> b6 = machine.Pin('B6') >>> b6 Pin(Pin.cpu.B6, mode=Pin.IN) >>> machine.I2C(1) I2C(1, scl=B6, sda=B7, freq=420000) >>> b6 Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, pull=Pin.PULL_UP, af=Pin.AF4_I2C1) >>> b6.init(machine.Pin.IN) >>> b6 Pin(Pin.cpu.B6, mode=Pin.ALT_OPEN_DRAIN, af=Pin.AF4_I2C1) With this commit the last print now works: >>> b6 Pin(Pin.cpu.B6, mode=Pin.IN)
2020-08-21docs: Change `\*` to `*` in argument lists.Zenix27
Latest versions of Sphinx (at least 3.1.0) do not need the `*` escaped and will render the `\` in the output if it is there, so remove it. Fixes issue #6209.
2020-08-21travis: Add zephyr build to CI.Maureen Helm
Adds a job to build the zephyr port in CI using the same docker container that the zephyr project uses for its own CI. Always make clean zephyr builds to ensure we don't just rebuild C code, but we also rebuild Kconfig and dts. This is required when switching between boards, which have different Kconfigs and device trees. Uses the tagged zephyr 2.3.0 release. Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-08-21zephyr: Include storage/flash_map.h unconditionally.Maureen Helm
Include storage/flash_map.h unconditionally so we always have access to the FLASH_AREA_LABEL_EXISTS macro, even if CONFIG_FLASH_MAP is not defined. This fixes a build error for the qemu_x86 board: main.c:108:63: error: missing binary operator before token "(" 108 | #elif defined(CONFIG_FLASH_MAP) && FLASH_AREA_LABEL_EXISTS(storage) | ^ ../../py/mkrules.mk:88: recipe for target 'build/genhdr/qstr.i.last' failed Signed-off-by: Maureen Helm <maureen.helm@nxp.com>
2020-08-12extmod/vfs_reader: Fix mp_reader_new_file to open file in "rb" mode.Damien George
mp_reader_new_file() is used to read in files for importing, either .py or .mpy files, for the lexer and persistent code loader respectively. In both cases the file should be opened in raw bytes mode: the lexer handles unicode characters itself, and .mpy files contain 8-bit bytes by nature. Before this commit importing was working correctly because, although the file was opened in text mode, all native filesystem implementations (POSIX, FAT, LFS) would access the file in raw bytes mode via mp_stream_rw() calling mp_stream_p_t.read(). So it was only an issue for non-native filesystems, such as those implemented in Python. For Python-based filesystem implementations, a call to mp_stream_rw() would go via IOBase and then to readinto() at the Python level, and readinto() is only defined on files opened in raw bytes mode. Signed-off-by: Damien George <damien@micropython.org>
2020-08-08tools/makemanifest.py: Print nicely formatted errors from mpy-cross.Damien George
If mpy-cross exits with an error be sure to print that error in a way that is readable, instead of a long bytes object. Signed-off-by: Damien George <damien@micropython.org>
2020-08-02py/persistentcode: Maintain root ptr list of imported native .mpy code.Damien George
On ports where normal heap memory can contain executable code (eg ARM-based ports such as stm32), native code loaded from an .mpy file may be reclaimed by the GC because there's no reference to the very start of the native machine code block that is reachable from root pointers (only pointers to internal parts of the machine code block are reachable, but that doesn't help the GC find the memory). This commit fixes this issue by maintaining an explicit list of root pointers pointing to native code that is loaded from an .mpy file. This is not needed for all ports so is selectable by the new configuration option MICROPY_PERSISTENT_CODE_TRACK_RELOC_CODE. It's enabled by default if a port does not specify any special functions to allocate or commit executable memory. A test is included to test that native code loaded from an .mpy file does not get reclaimed by the GC. Fixes #6045. Signed-off-by: Damien George <damien@micropython.org>
2020-07-26tests/micropython: Improve .mpy import tests to run on more targets.Damien George
All imports are now tested to see if the test should be skipped, UserFile.read is removed, and UserFile.readinto is made more efficient. Signed-off-by: Damien George <damien@micropython.org>
2020-07-26tests: Move .mpy import tests from import/ to micropython/ dir.Damien George
These tests are specific to MicroPython so have a better home in the micropython/ test subdir, and putting them here allows them to be run by all targets, not just those that have access to the local filesystem (eg the unix port). Signed-off-by: Damien George <damien@micropython.org>
2020-07-26tools/makemanifest.py: Use errno.EEXIST instead of number 17.Damien George
To make this code more portable, across different platforms. Signed-off-by: Damien George <damien@micropython.org>
2020-07-25extmod/uasyncio: Add StreamReader.readexactly(n) method.Damien George
It raises on EOFError instead of an IncompleteReadError (which is what CPython does). But the latter is derived from EOFError so code compatible with MicroPython and CPython can be written by catching EOFError (eg see included test). Fixes issue #6156. Signed-off-by: Damien George <damien@micropython.org>
2020-07-25stm32/usbdev: Fix calculation of SCSI LUN size with multiple LUNs.Damien George
The SCSI driver calls GetCapacity to get the block size and number of blocks of the underlying block-device/LUN. It caches these values and uses them later on to verify that reads/writes are within the bounds of the LUN. But, prior to this commit, there was only one set of cached values for all LUNs, so the bounds checking for a LUN could use incorrect values, values from one of the other LUNs that most recently updated the cached values. This would lead to failed SCSI requests. This commit fixes this issue by having separate cached values for each LUN. Signed-off-by: Damien George <damien@micropython.org>
2020-07-25py/compile: Don't await __aiter__ special method in async-for.Jonathan Hogg
MicroPython's original implementation of __aiter__ was correct for an earlier (provisional) version of PEP492 (CPython 3.5), where __aiter__ was an async-def function. But that changed in the final version of PEP492 (in CPython 3.5.2) where the function was changed to a normal one. See https://www.python.org/dev/peps/pep-0492/#why-aiter-does-not-return-an-awaitable See also the note at the end of this subsection in the docs: https://docs.python.org/3.5/reference/datamodel.html#asynchronous-iterators And for completeness the BPO: https://bugs.python.org/issue27243 To be consistent with the Python spec as it stands today (and now that PEP492 is final) this commit changes MicroPython's behaviour to match CPython: __aiter__ should return an async-iterable object, but is not itself awaitable. The relevant tests are updated to match. See #6267.
2020-07-25docs/esp32: Fix machine.Timer quickref to specify HW timers.Josh Lloyd
Also remove trailing spaces on other lines.
2020-07-25docs/library: Update pyb.UART to correct pyboard UART availability.Howard Lovatt
On original pyboard UART 5 isn't available; added pyboard D availability.
2020-07-25docs/library: Update pyb.Timer to add missing args and defaults to init.Howard Lovatt
2020-07-25docs/library: Update pyb.SPI init method to add descr about "ti" arg.Howard Lovatt
2020-07-22nrf: Enable more features for all targets.Glenn Ruben Bakke
Enabling the following features for all targets, except for nrf51 targets compiled to be used with SoftDevice: - MICROPY_PY_ARRAY_SLICE_ASSIGN - MICROPY_PY_SYS_STDFILES - MICROPY_PY_UBINASCII
2020-07-22nrf: Split mpconfigport.h into multiple files.Glenn Ruben Bakke
Splitting mpconfigport.h into multiple device specific files in order to facilitate variations between devices. Due to the fact that the devices might have variations in features and also variations in flash size it makes sense that some devices offers more functionality than others without being limited by restricted devices. For example more micropython features can be activated for nrf52840 with 1MB flash, compared to nrf51 with 256KB.
2020-07-22lib/libc: Fix string0's implementation of strncpy.Damien George
Fixing 98e583430fb7b793119db27bad9f98119e81579f, the semantics of strncpy require that the remainder of dst be filled with null bytes. Signed-off-by: Damien George <damien@micropython.org>
2020-07-21lib/libm_dbl: Add round.c source code.Zoltán Vörös
This code is imported from musl, to match existing code in libm_dbl. The file is also added to the build in stm32/Makefile. It's not needed by the core code but, similar to c5cc64175be32cb1e4f3f1a249667bc9f5a12613, allows round() to be used by user C modules or board extensions.
2020-07-21py/obj.h: Fix mp_seq_replace_slice_no_grow to use memmove not memcpy.Damien George
Because the argument arrays may overlap, as show by the new tests in this commit. Also remove the debugging comments for these macros, add a new comment about overlapping regions, and separate the macros by blank lines to make them easier to read. Fixes issue #6244. Signed-off-by: Damien George <damien@micropython.org>
2020-07-21tests/basics: Split out memoryview slice-assign tests to separate file.Damien George
Signed-off-by: Damien George <damien@micropython.org>
2020-07-21lib/stm32lib: Update library for H7 v1.6.0 and WB v1.6.0.Damien George
Changes in this new library version are: - Update H7 HAL to v1.6.0. - Update WB HAL to v1.6.0. - Add patches to fix F4 ll_uart clock selection for UART9/UART10. Signed-off-by: Damien George <damien@micropython.org>
2020-07-21esp32/network_lan: Add support for IP101 PHY.Kenneth Ryerson
Signed-off-by: Kenneth Ryerson <kenneth.ryerson@gmail.com>
2020-07-21docs/library: Update documentation of esp32's RMT.Jonathan Hogg
This explains how looping now works, and removes the warning about calling wait_done().
2020-07-21esp32/esp32_rmt: Properly fix looping behaviour of RMT.Jonathan Hogg
A previous commit 3a9d948032e27f690e1fb09084c36bd47b1a75a0 can cause lock-ups of the RMT driver, so this commit reverses that, adds a loop_en flag, and explicitly controls the TX interrupt in write_pulses(). This provides correct looping, non-blocking writes and sensible behaviour for wait_done(). See also #6167.
2020-07-21extmod/modussl_mbedtls: Integrate shorter error strings.Thorsten von Eicken
The stm32 and esp32 ports now use shorter error strings for mbedtls errors. Also, MBEDTLS_ERROR_C is enabled on stm32 by default to get these strings.
2020-07-20lib/mbedtls_errors: Add code to patch mbedtls for shortened error strs.Thorsten von Eicken
The file `mbedtls_errors/mp_mbedtls_errors.c` can be used instead of `mbedtls/library/error.c` to give shorter error strings, reducing the build size of the error strings from about 12-16kB down to about 2-5kB.
2020-07-20lib/libc: Add implementation of strncpy.Thorsten von Eicken
2020-07-20extmod/modussl: Improve exception error messages.Thorsten von Eicken
This commit adds human readable error messages when mbedtls or axtls raise an exception. Currently often just an EIO error is raised so the user is lost and can't tell whether it's a cert error, buffer overrun, connecting to a non-ssl port, etc. The axtls and mbedtls error raising in the ussl module is modified to raise: OSError(-err_num, "error string") For axtls a small error table of strings is added and used for the second argument of the OSErrer. For mbedtls the code uses mbedtls' built-in strerror function, and if there is an out of memory condition it just produces OSError(-err_num). Producing the error string for mbedtls is conditional on them being included in the mbedtls build, via MBEDTLS_ERROR_C.
2020-07-20docs/library: For ubluetooth, add docs for _IRQ_GATTS_INDICATE_DONE.Jim Mussared
2020-07-20extmod/modbluetooth: Add event for "indicate acknowledgement".Jim Mussared
This commit adds the IRQ_GATTS_INDICATE_DONE BLE event which will be raised with the status of gatts_indicate (unlike notify, indications require acknowledgement). An example of its use is added to ble_temperature.py, and to the multitests in ble_characteristic.py. Implemented for btstack and nimble bindings, tested in both directions between unix/btstack and pybd/nimble.
2020-07-20extmod/modbluetooth: Fix so it builds in peripheral-only mode.Jim Mussared