Endurance testing with python
Overview
The following information covers some of the challanges when doing power cycle endurance testing where the unit under test (UUT) is connected to the python controlling application using a serial port. Some of the challenges include not letting the extranous data that is often received over serial as the UUT powers off and back on. These characters can trip errors in the python Unicode decoder that need to be ignored.
Links
- https://pypi.python.org/pypi/pexpect
- http://pexpect.sourceforge.net/fdpexpect.html
- http://helpful.knobs-dials.com/index.php/Python_usage_notes/pty_and_pexpect
- https://cuddonnet.blogspot.com/2015/04/python-x10-cm19a-usb-driver-software.html
Power cycle setup
The key components involved include:
- Desktop or laptop PC running Linux. I used a FT2232 type USB to serial dongle.
- Unit under test (UUT) - embedded Linux device being tested.
- Power cycling hardware. I like X10, which consists of
The only downside I have with X10 is you can't turn it off and back on quickly - something on the order of 3 seconds minimum. I like X10 because I have lots of applicance modules so I can run multiple tests at the same time, I can use it with desktop icons so I don't have to physically power cycle hardware, and because the appliance module can be connected to any outlet, I can set the endurance test hardware out of the way.
python X10 control
There are three subtle host PC configuration steps that I found to be needed to allow X10 power control to work smoothly. Hopefully future Linux distros will do a better job supporting X10 modules.
- Unplug CM19a X10 Transceiver
- Black list other drivers that attempt to own the X10 USB RF tranceiver
echo -e "#Blacklist to enable CM19a driver\nblacklist lirc_ati\nblacklist ati_remote" | \
sudo tee -a /etc/modprobe.d/blacklist.conf
sudo modprobe --remove lirc_atiusb ati_remote
- Allow all users to interact with the X10 USB RF tranceiver
echo -e "# Allow all users to read and write to the CM19a X10 Transceiver (USB)\nSYSFS{idVendor}==\"0bc7\", SYSFS{idProduct}==\"0002\", MODE=\"666\"" | \
sudo tee /etc/udev/rules.d/cm19a.rules
- Plug in the X10 USB RF tranceiver
- Remove old version of python USB module
sudo apt-get remove python-usb
Install updated python USB module
wget http://softlayer-dal.dl.sourceforge.net/project/pyusb/PyUSB%201.0/1.0.0-alpha-3/pyusb-1.0.0a3.tar.gz tar -xzf pyusb-1.0.0a3.tar.gz cd pyusb-1.0.0a3/ python setup.py build sudo python setup.py install
- Unfortunately, the X10 USB RF tranceiver python code doesn't come with the standard setup.py, so I ended up installing it in the same directory tree as my python expect scripts resided.
mkdir $HOME/work/pexpect cd $HOME/work/pexpect wget http://ubuntuone.com/0O4fjPSt2QFAY8yNcdxZVz unzip CM19aDriver_v3.0.zip
Now you can test by turning a module on and off. My module is set to house code G unit code 3.
./CM19aDriver.py G 3 ON ./CM19aDriver.py G 3 OFF
python expect setup
The good news is the is an expect package for python called pexpect. If you have used any of the other variations of expect, you will find the python version works as you would expect.
I added my Linux user ID to the dialout group so I could send and receive data or the serial ports and also installed the pexpect package.
sudo addgroup $USER dialout sudo apt-get python-pexpect
Ignoring binary data
The bad news is I found pexpect to assume it was working with a text (not binary) data stream. When binary data was received, the python exception was so low that I couldn't figure out how to easily ignore it while waiting for the expected data. In my case, the binary data was due to powering off and on devices connected over RS-232 serial. Occatiionally I would get a random binary pattern during a power cycle and python said it could decode the Unicode character. Thanks to Open Source code, I could at least work around the issue.
I ended up changing the pexpect 2.5.1 source code to solve this issue. Specifically, I made the following changes to /usr/local/lib/python2.7/dist-packages/pexpect/__init__.py. The changes basically told python to ignore unicode decode error and to strip non-ASCII from strings before logging them.
--- pexpect-u-2.5.1/pexpect/__init__.py 2011-12-10 15:48:07.000000000 -0700
+++ ../pexpect__init__.py 2013-10-01 11:13:59.218765248 -0600
@@ -152,7 +152,7 @@
def _cast_unicode(s, enc):
if isinstance(s, bytes):
- return s.decode(enc)
+ return s.decode(enc, 'ignore')
return s
re_type = type(re.compile(''))
@@ -864,7 +864,7 @@
self.flag_eof = True
raise EOF ('End Of File (EOF) in read_nonblocking(). Empty string style platform.')
- s2 = self._cast_buffer_type(s)
+ s2 = "".join(i for i in self._cast_buffer_type(s) if ord(i)<128)
if self.logfile is not None:
self.logfile.write(s2)
self.logfile.flush()
@@ -1620,12 +1620,12 @@
def _prepare_regex_pattern(self, p):
"Recompile bytes regexes as unicode regexes."
if isinstance(p.pattern, bytes):
- p = re.compile(p.pattern.decode(self.encoding), p.flags)
+ p = re.compile(p.pattern.decode(self.encoding, 'ignore'), p.flags)
return p
def read_nonblocking(self, size=1, timeout=-1):
return super(spawn, self).read_nonblocking(size=size, timeout=timeout)\
- .decode(self.encoding)
+ .decode(self.encoding, 'ignore')
read_nonblocking.__doc__ = spawnb.read_nonblocking.__doc__
In looking at the latest pexpect code at git@gitorious.org:pexpect/pexpect.git, there is no direct call to decode(), but they logging change may still be useful.
python endurance testing example
# There are two parts to this code - one is the X10 power switch logic
# to power the board on and off. The other is the expect logic to monitor
# the boot process.
# To configure the port, first run
# stty -F /dev/ttyUSB5 115200 cs8 -clocal -crtscts -parenb -inpck -ixoff -ixon
import os, sys, time
from datetime import datetime
# System dependent constants
cm19adriverdir='/projects/x10'
#devport="/dev/ttyUSB5"
prompt = "# "
rootpassword="foobar"
x10housecode="G"
x10unitcode="3"
# delay in seconds
poweroffdelay=5
# set bootlog to sys.stdout or file("boot.log", "a")
bootlog=file("boot.log", "a")
import pexpect
import fdpexpect
homedir=os.environ['HOME']
sys.path.append(homedir + cm19adriverdir)
import CM19aDriver
# Initialize power switch
cm19a = CM19aDriver.CM19aDevice()
banner=time.strftime("---------------%Y-%m-%d %H:%M:%S---------------\n", time.localtime())
bootlog.write(banner)
print banner
# returns number of errors encountered (zero or one)
def wait_for(m, msg,tm=60) :
try:
m.expect(msg,timeout=tm)
return 0
except pexpect.TIMEOUT :
return 1
except pexpect.EOF :
# sometimes the /dev/tty* UART port reports EOF, just ignore
return 0
def init_expect() :
tm = int(time.mktime(time.localtime()))
btlog = file("boot-" + repr(tm) + ".log", "w")
fd = os.open(devport, os.O_RDWR|os.O_NOCTTY )
m = fdpexpect.fdspawn(fd, timeout=60, logfile=btlog)
m.setecho(False)
return m
def power_on(m) :
result = cm19a.send(x10housecode, x10unitcode, "ON")
try:
# verify we get init running
wait_for(m, " login: ", tm=180)
m.sendline("root")
wait_for(m, "Password: ")
m.sendline(rootpassword)
wait_for(m, ":~# ")
except (pexpect.EOF, OSError) :
print "Serial port error"
result = cm19a.send(x10housecode, x10unitcode, "OFF")
return 1
return 0
def power_off(m) :
cm19a.send(x10housecode, x10unitcode, "OFF")
def run_remote_command(m, cmd) :
m.sendline(cmd);
wait_for(m, cmd)
wait_for(m, prompt)
print m.before.strip()
def endurance_test_power_cycle() :
loops = 0
m = init_expect()
while True :
try:
print "LOOP COUNT %d" % loops
power_on(m)
time.sleep(poweroffdelay)
power_off(m)
loops += 1
except KeyboardInterrupt :
print
print "Keyboard interrupt:"
print " LOOP COUNT %d" % loops
cm19a.send(x10housecode, x10unitcode, "OFF")
return
except (pexpect.EOF, OSError) :
# sometimes the /dev/tty* UART port reports EOF, just ignore
print "uart error:"
if __name__ == '__main__' :
# get to known state
result = cm19a.send(x10housecode, x10unitcode, "OFF")
time.sleep(poweroffdelay)
endurance_test_power_cycle
TP Link HS100 HS101
The GadgetReactor pyHS100 python3 Open Source project provides a nice CLI tool. I fought problems with PYTHONPATH to get pyhs100 to work on my Mac.
First, configure your HS100 using KasaSmart phone app. Once the HS100 is able to connect to your wifi access point, you can use pyhs100 to automate control.
export PYTHONPATH= pip3 install pyHS100 pyhs100
The pyhs100 CLI will search for your HS100. I got the output:
No host name given, trying discovery.. Discovering devices for 3 seconds == TP Switch - HS100(US) == OFF Host/IP: 10.111.0.71 LED state: True On since: 2019-09-13 11:18:18.256648 == Generic information == Time: 2019-09-13 11:18:18 Hardware: 1.0 Software: 1.1.1 Build 160725 Rel.163650 MAC (rssi): 50:C7:BF:9B:AC:C7 (-68)
Then you can control the switch:
pyhs100 --host 10.111.0.71 --plug on pyhs100 --host 10.111.0.71 --plug off
It took 20 seconds to turn the plug on and off 100 times. there was an odd pause after every 6 power cycles.
Old stuff to be ignored
- Unfortunately, the X10 USB RF tranceiver python code doesn't come with the standard setup.py, so installing the module is a manual process
wget http://ubuntuone.com/0O4fjPSt2QFAY8yNcdxZVz
unzip CM19aDriver_v3.0.zip
cat <<EOF >setup.py
#!/usr/bin/env python
from distutils.core import setup
setup(
name='cm19a',
version='3.0',
description='Python CM19A X10 USB RF tranceiver controller',
author='Andrew Cuddon',
author_email='andrew@cuddon.net',
license = 'Unknown',
url='http://www.cm19a.com/2013/02/python-x10-cm19a-usb-software-linux.html'
packages=['cm19a'],
long_description =
"""
CM19aDriver offers easy CM19A X10 USB tranceiver control in Python.
It requires a newer version of pyusb.
"""
)
EOF
python setup.py build
sudo python setup.py install