2012-10-23 Andy Doan <email address hidden>
string_to_list(boot_cmds) not working on older python
The version of python we are running in production hits a problem
related to our new use of configglue. configglue passes a value
like "boot_cmds" as a unicode string. Our string_to_list implementation
on control isn't able to convert this to a list.
2012-10-23 Andy Doan <email address hidden>
post release bump
2012-10-22 Andy Doan <email address hidden>
release-0.19.2
fix openembedded file permission issue
2012-10-22 Andy Doan <email address hidden>
release-0.19.1
Two setup.py sdist issues:
1) I forgot to check in the action lava-test-runner.init.d file
2) setup.py wasn't pulling in lava_test_shell files
2012-10-22 Andy Doan <email address hidden>
post release bump
2012-10-22 Andy Doan <email address hidden>
release-0.19
2012.10
2012-10-22 Andy Doan <email address hidden>
fixes for lava-test-shell
2012-10-22 Andy Doan <email address hidden>
move the workfile before the test starts
This is a bit smarter than the initial approach of doing it afterwards
2012-10-19 Andy Doan <email address hidden>
fix weird shutdown issue with lava-test-shell on fastmodel
I hit a weird issue with lava-test-shell doing Ubuntu based
FastModels. Even though we call sync, things weren't getting flushed
to the sd.img file properly.
This just adds a halt command to the _power_off logic to try and
make sure things halt as cleanly as possible.
2012-10-18 Andy Doan <email address hidden>
clean up the CommandRunner logic a bit in target.py
2012-10-18 Andy Doan <email address hidden>
have lava-test-shell try and clean up after itself
We want to try and remove our test-runner.conf file so that
the job won't get executed on the next boot. This is needed in
case a job file includes a lava_test action after the lava_test_shell
action.
In order to do this, the test-runner.conf can't be in /etc for android
because this partition is mounted read-only. So everything has been
moved to a read-write partition except for the etc stuff needed to
launch our test-runner at startup.
2012-10-18 Andy Doan <email address hidden>
fix dumb mistake from revno 408
2012-10-18 Andy Doan <email address hidden>
fix dumb mistake from previous commit
2012-10-18 Andy Doan <email address hidden>
fix health check regression for master images
Target implementations must now always set self.deployment_data. We
were missing logic to set this when we find cached tarballs for health
jobs. This commit does two things:
1. Add "data" to the tuple that gets cached. This data is flexible, but
in practice is just "oe" or "ubuntu" to tell us which deployment_data
is needed.
2. Make tarballcache more resilient. This changes how tarballcache is
set up. So current installations not including this new "data" file will
need to re-generate their tarballs. While I was at it, I also fixed
logic so that accidental deletion of say "boot.tgz" can be managed
properly (ie - it will clear the cache and regenerate)
2012-10-17 Andy Doan <email address hidden>
FastModel: add new support for foundation models
2012-10-17 Andy Doan <email address hidden>
add oe data for lava_test_shell
This adds an init.d approach that can work for OE images.
TODO - this doesn't support a "deps" section in the test defintion.
Can we do something like this for OE?
2012-10-17 Andy Doan <email address hidden>
update our _customize_ubuntu logic
We now have the need to customize OE images and also provide target
data for them. We could require a flag in our job files to denote if
its OE or Ubuntu. However, in practice our OE changes are pretty much
standard linux stuff and our Ubuntu changes take deal with things like
Dash/Upstart.
This keys off the existence of /etc/debian_version to help make the
determination.
2012-10-17 Andy Doan <email address hidden>
minor change to get ricardos patch to run
just a minor syntax thing
2012-10-17 Ricardo Salveti de Araujo <email address hidden>
FastModel: adding support for the foundation model
2012-10-16 Andy Doan <email address hidden>
all power_off logic requires a sync
We missed a case where we weren't doing a sync before calling power_off.
This was causing results for FastModels to not get flushed to disk.
2012-10-16 Andy Doan <email address hidden>
regression with lava-test for fastmodel
I'm not sure how this ever worked, but the recent juggling around
of prompts has indicated an issue with the default timeout set
for fastmodels. Any command that passes -1 as a timeout uses the
default which was 90, and way too small to do something like run
the bigLITTLE suite. This gets it to a sane value used by all
other device types.
2012-10-16 Andy Doan <email address hidden>
ensure fastmodel uses lo for network interface
In the old implementation the client had a function,
get_android_adb_interface. This doesn't work well with the new
target approach. And we really already had a way of handling this
just using the config for the model.
2012-10-16 Andy Doan <email address hidden>
regression in fastmodel android from latest target refactor
Somehow I missed the ping alias for fastmodels. I also re-did
how we write this file. The logging_system way was a bit overkill
2012-10-17 Michael Hudson-Doyle <email address hidden>
Change the setting of the shell prompts -- instead of setting it after the test
image boots, just have deployment force the prompt to be what we want.
Also make the master image code a bit more single-purpose.
2012-10-16 Michael Hudson-Doyle <email address hidden>
cleanups
2012-10-16 Michael Hudson-Doyle <email address hidden>
remove tester_hostname config option
2012-10-16 Michael Hudson-Doyle <email address hidden>
remove another now gratuitous change
2012-10-16 Michael Hudson-Doyle <email address hidden>
TesterCommandRunner no longer needs to inherit from NetworkCommandRunner
2012-10-16 Michael Hudson-Doyle <email address hidden>
merge trunk
2012-10-16 Michael Hudson-Doyle <email address hidden>
leave reliable_session alone for now
2012-10-16 Michael Hudson-Doyle <email address hidden>
remove dead line
2012-10-16 Michael Hudson-Doyle <email address hidden>
testing fixes
2012-10-15 Michael Hudson-Doyle <email address hidden>
oops
2012-10-15 Michael Hudson-Doyle <email address hidden>
make master_ip clearing work like trunk again
2012-10-15 Michael Hudson-Doyle <email address hidden>
wait for device to boot in boot_linaro_image; wait for network in reliable_session()
2012-10-15 Michael Hudson-Doyle <email address hidden>
pass image to _customize_ubuntu
2012-10-15 Michael Hudson-Doyle <email address hidden>
remove debugging
2012-10-15 Michael Hudson-Doyle <email address hidden>
remove _power_on/power_on split again
2012-10-15 Michael Hudson-Doyle <email address hidden>
unrelated bug
2012-10-15 Michael Hudson-Doyle <email address hidden>
set default shell PS1 when deploying ubuntu for realz this time
2012-10-15 Michael Hudson-Doyle <email address hidden>
just plain set PS1 for ubuntu deployments too
2012-10-15 Michael Hudson-Doyle <email address hidden>
push prompt setting into power_on()
2012-10-15 Michael Hudson-Doyle <email address hidden>
quick 360 to always use deployment_data to determine PS1 to look for
2012-10-15 Michael Hudson-Doyle <email address hidden>
not sure why i removed that
2012-10-15 Michael Hudson-Doyle <email address hidden>
bugs found by andy's torture script
2012-10-12 Michael Hudson-Doyle <email address hidden>
add a comment
2012-10-12 Michael Hudson-Doyle <email address hidden>
stick INITIAL_TESTER_PS1 in the deployment data
2012-10-12 Michael Hudson-Doyle <email address hidden>
fixes
2012-10-12 Michael Hudson-Doyle <email address hidden>
missing some bits
2012-10-12 Michael Hudson-Doyle <email address hidden>
* for ubuntu deployments, set /etc/hostname to the value of tester_hostname for the device
* look for root@$tester_hostname on boot, then clobber an explicit PS1 out
* for android, just clobber PS1 when configuring the image
2012-10-12 Michael Hudson-Doyle <email address hidden>
trivialities
2012-10-12 Michael Hudson-Doyle <email address hidden>
in_test_shell is a private method now
2012-10-12 Michael Hudson-Doyle <email address hidden>
copy CommandRunner.run to MasterCommandRunner, remove some stupid bits
2012-10-12 Michael Hudson-Doyle <email address hidden>
small drive by things
2012-10-12 Michael Hudson-Doyle <email address hidden>
remove idiocy
2012-10-12 Michael Hudson-Doyle <email address hidden>
have the deploy_* methods explicitly boot into the master image
2012-10-12 Michael Hudson-Doyle <email address hidden>
restore setting of master_ip
2012-10-12 Michael Hudson-Doyle <email address hidden>
just plain force the PS1 in the master image to be what we want, only look at the config value for the first prompt after boot
2012-10-15 Yongqin Liu <email address hidden>
update the sed command for mount sdcard of android image with the mount point of /storage/sdcard0
2012-10-15 Yongqin Liu <email address hidden>
update the sed command for sdcard mount
2012-10-15 Yongqin Liu <email address hidden>
fix the typo of the calling _hard_reboot
2012-10-15 Yongqin Liu <email address hidden>
fix type of calling of _hard_reboot
2012-10-10 Andy Doan <email address hidden>
more documentation updates
2012-10-10 Andy Doan <email address hidden>
update usage.rst page
2012-10-10 Andy Doan <email address hidden>
doc fixes
updated index page
removed unuseful and hard to maintain pages: code.rst and todo.rst
fixed configuration page to just show what it was pointing to
2012-10-10 Andy Doan <email address hidden>
update our documentation
This was getting pretty out of date. It also includes help on the newest
lava_test_shell testing format.
2012-10-09 Michael Hudson-Doyle <email address hidden>
fix MasterImageTarget.file_system to propagate deletes from host to target
2012-10-09 Michael Hudson-Doyle <email address hidden>
idiot hudson
2012-10-09 Michael Hudson-Doyle <email address hidden>
in MasterImageTarget.file_system, create the tarball from above the target directory
and remove the target directory before untarring
this makes file deletion propagate
i changed the tarball location so that the permission bits of the target directory are hopefully preserved
2012-10-04 Andy Doan <email address hidden>
support lava_test_shell action
2012-10-04 Andy Doan <email address hidden>
more fixes for from code review
improve some exception handling in the bundling logic
be explicit about what files to copy to the test results
2012-10-03 Andy Doan <email address hidden>
allow lava_test_shell to work off filesystem rather than tarfile
This originally was done via a tarball because I wasn't sure how the
testshell code would get to interact with the target device. however,
now that we have a clear API there's no need for this hackery.
Also make mem/cpu info not required. There's no strong reason to enfore
this - ie its better to get what bundle results are avaiable than nothing
at all.
2012-10-03 Andy Doan <email address hidden>
fix minor comments from review
* fix subprocess.check_call to not use shell in repo clones
* iterate repos with less indentation
* clean up docstrings
2012-10-03 Andy Doan <email address hidden>
ensure lava-test-shell attachments are really copied
The old code wasn't actually copying run.sh. We also should copy
install.sh.
2012-10-03 Andy Doan <email address hidden>
clean up code for jobs with multiple lava_test_shell actions
The current code, just sort of worked by piling up the results across
lava_test_shell actions and parsing them all each time. This puts
each one in its own unique bundle file that the submit results logic
can then work with.
2012-10-03 Andy Doan <email address hidden>
allow action to handle multiple testdef urls
The lava_test_shell code had this logic already, but the action didn't.
This changes the action to accept an array of URLs. This implied some
fixes to badness in the code:
* making the /lava/tests/<testname> handle repeats by now using an
index so its /lava/tests/01_testname, etc
* configure_target decides the name of the test install directory,
explains how the target's path and the hosts path can differ, and
then passes this path to its functions.
2012-10-03 Andy Doan <email address hidden>
fix lava-test-shell interpreter
This started as /system/bin/mksh for android, then was hacked around
by calling via "sh" in each runner. This allows the lava_test_shell
action to determine the proper interpreter and allow us to call it
properly in our scripts.
2012-10-02 Andy Doan <email address hidden>
fix some bugs in the MP
2012-10-02 Andy Doan <email address hidden>
add bundle building logic to lava_test_shell
This adds the logic to take the test results from the lava-test-shell
logic and produce a properly formatted lava-dashboard bundle
2012-10-02 Andy Doan <email address hidden>
add install support to lava_test_shell
This handles the install part of declarative test in the action.
2012-10-01 Andy Doan <email address hidden>
add run support to lava_test_shell
This adds logic to build the run.sh script based on the run section
of the test definition
2012-10-01 Andy Doan <email address hidden>
add notion of testdef to lava_test_shell action
This adds a testdef_url and timeout to the parameters this action
accepts. The testdef_url should point to a file that supports the
lp:lava-test's "declarative test" model. The timeout value is how
long in seconds the test will be given to run.
This series just gets the testdef in place. The next few patches
actually do something with this.
An example declarative test would be:
{
"format": "Lava-Test Test Definition 1.0",
"test_id": "passfail",
"install": {
"bzr-repos": ["lp:lava-test"],
"git-repos": ["git://git.linaro.org/ubuntu/daily-prebuilt-images.git"],
"steps": ["echo installing!"]
},
"run": {
"steps": ["/bin/echo mem-stress-a7 : PASS"]
},
"parse": {
"pattern": "(?P<test_case_id>.*-*)\\s+:\\s+(?P<result>(PASS|FAIL))",
"fixupdict": {
"PASS": "pass",
"FAIL": "fail"
}
}
}
2012-10-01 Andy Doan <email address hidden>
include new lava-test-shell scripts
These are the first stage for the basis of a more black box style approach
to testing in LAVA. There are two scripts:
* lava-test-runner(andrdoid/ubuntu): This script is intended to be
launched near the end of the boot process. Its job is to find tests
that have been configured to work. It does so by looking at the
${WORKFILE} for tests defined to run. Each test directory must
include a "run.sh" file which is to be run by the lava-test-shell.
The test runner also saves the information needed later on for the hardware
and software context information that goes into a LAVA bundle.
* lava-test-shell: This is super simple at the moment. Its responsible
for executing the actual command. It streams stdout/stderr to both the
serial console and to a file. Currently stderr is streamed to stdout.log
and this should be fixed, but its a start. We also save the exit code
for the process.
2012-10-01 Andy Doan <email address hidden>
workaround bug in python's tarfile implementation
Our version of python has an issue with unicode filenames in a tarball.
We had old code that just shelled out to "tar -xzf" and that seems like
the safest bet for the time being.
2012-10-01 Andy Doan <email address hidden>
convert to using new device.target API for devices
This converts the dispatcher over to using a more limited, well-defined
approach for dealing with actual device types. It then keeps the "client"
part of the implementation down to just what's required by current actions
in the dispatcher.
2012-10-01 Andy Doan <email address hidden>
code review updates
This addresses review notes for the original MP.
master.py:
* ubuntu boot_cmds was set wrong
* use == over "is" in file_system method
* let SimpleHTTPServer pick its own port
* fix regex for finding port number (pexpect uses non-greedy matching)
device-defaults.conf:
* we already gave soft_boot_cmd a default in config.py
base.py:
* use a common empty_buffer function
* use " over ' for docstring and fix grammar
target.py:
* use " over ' for docstring
* explain partition used for file_system method
* increase the timeout for the "sync" command in power off to something
larger, but still arbitrary
2012-09-30 Andy Doan <email address hidden>
fix deployment data concept in Target
The original approach in this patchset with a single dictionary
was pretty clumsy. This approach allows other modules to register
things in a more flexible way that makes accessing the information
better.
2012-09-30 Andy Doan <email address hidden>
create directory if missing on calls to target.file_system
Create the target directory if the caller is depending on it existing
2012-09-30 Andy Doan <email address hidden>
convert master client to target based
2012-09-30 Andy Doan <email address hidden>
support android-deploy in master image device
2012-09-30 Andy Doan <email address hidden>
add master device support for hwpack+rfs
This adds in the logic to run l-m-c for Ubuntu images to the master
image device implementation.
Tested with snippet:
from lava_dispatcher.device.master import MasterImageTarget
from lava_dispatcher.config import get_config, get_device_config
from lava_dispatcher.context import LavaContext
config = get_config("lava-dispatcher", "/srv/lava/instances/staging/etc/lava-dispatcher")
ctx = LavaContext('panda01', config, None, None)
device_config = get_device_config('panda01', config.config_dir)
target = MasterImageTarget(ctx, device_config)
target.deploy_linaro('http://snapshots.linaro.org/precise/hwpacks/panda/304/hwpack_linaro-panda_20120921-304_armhf_supported.tar.gz', 'http://snapshots.linaro.org/precise/images/nano/412/linaro-precise-nano-20120921-412.tar.gz')
2012-09-30 Andy Doan <email address hidden>
add pre-built flashing logic to master device
This also introduces a new method (tarballcache) for manage our cached
tarballs for health jobs. Our caching logic in the client's master.py
was complex and could have race conditions. This simplifies things quite
a bit and should actually work better.
doanac's axiom: Those who devise their own synchronization algorithms
are bound to lose the 'race'.
testing done with script:
from lava_dispatcher.device.master import MasterImageTarget
from lava_dispatcher.config import get_config, get_device_config
from lava_dispatcher.context import LavaContext
config = get_config("lava-dispatcher", "/srv/lava/instances/staging/etc/lava-dispatcher")
ctx = LavaContext('panda01', config, None, None)
device_config = get_device_config('panda01', config.config_dir)
target = MasterImageTarget(ctx, device_config)
target.deploy_prebuilt('http://snapshots.linaro.org/precise/pre-built/lt-panda/292/lt-panda-precise_nano_20120720-292.img.gz')
2012-09-30 Andy Doan <email address hidden>
implement file_system support in master image device
2012-09-30 Andy Doan <email address hidden>
add on/off function to master image device
This allows us to boot into the linaro image. It might make more
sense to do the "deploy_image" patches before this, but they are hard
to verify without boot support. So this was done first.
These functions were copied from the old master.py client. However,
the _boot_linaro_image logic was updated to support ubuntu/android in
one function rather than two functions. This is done by introducing a
new dictionary to the target device called "deployment_data". This is
used to set per-deployment options. For example, when the deploy_*
methods are implemented for master.py they will set the proper boot_cmds.
2012-09-30 Andy Doan <email address hidden>
create masterimage target device skeleton
this just gives us the ability to boot into the master image
2012-09-30 Andy Doan <email address hidden>
make qemu device a generic client implementation
Make the qemu client be based on the TargetBasedClient
2012-09-30 Andy Doan <email address hidden>
make fastmodel device a generic client implementation
The fastmodel client is now basically a generic shell that works with
a Target object. This makes it totally generic so that new devices built
completely on the Target API can still work with tradition actions in
the dispatcher
We also remove the old fastmodel_deploy action which was never used in
production.
2012-09-30 Andy Doan <email address hidden>
move test_data_attachment logic into target device
This will be needed for target device implementations and is not specific
to our traditional lava-test oriented clients
2012-09-30 Andy Doan <email address hidden>
convert fastmodel code over to new target device interface
This moves everything not needed by the traditional dispatcher client
code into the device implementation.
2012-09-28 Andy Doan <email address hidden>
add concept of a target device
The lava-test-shell work allows us to have a much more narrow view about
interacting with a target device. This commit defines the minimum interface
required for a lava-test-shell based action:
* power_on
* power_off
* deploy_* - deploy an image type to the target
* file_sytem - allows interaction with the files on the target
2012-10-01 Andy Doan <email address hidden>
configglue bug fixes found while doing fastmodel.py testing
2012-09-27 Andy Doan <email address hidden>
configglue bug fix for fastmodel.py
2012-10-01 Andy Doan <email address hidden>
ensure we delete the host_dir
2012-09-27 Andy Doan <email address hidden>
ensure we delete the host_dir
While doing the previous commit, I noticed that the temp directory
created wasn't being deleted. Upon further investigation I noticed
this also results in littering our /tmp directory with tons of result
bundles from Android runs. This is a quick fix.
2012-09-27 Antonio Terceiro <email address hidden>
do not set tester_hostname to empty string in default config
2012-09-27 Michael Hudson-Doyle <email address hidden>
use configglue to define a schema for our configuration
buyer beware: this may be a little disruptive, test before upgrading!
2012-09-26 Michael Hudson-Doyle <email address hidden>
grr
2012-09-26 Michael Hudson-Doyle <email address hidden>
no, there is always more
2012-09-26 Michael Hudson-Doyle <email address hidden>
final (?) touches
2012-09-26 Michael Hudson-Doyle <email address hidden>
even more mechanical progress
2012-09-26 Michael Hudson-Doyle <email address hidden>
more
2012-09-26 Michael Hudson-Doyle <email address hidden>
more mechanical progress
2012-09-26 Michael Hudson-Doyle <email address hidden>
random progress
2012-09-26 Michael Hudson-Doyle <email address hidden>
more working
2012-09-26 Michael Hudson-Doyle <email address hidden>
works a little bit now
2012-09-26 Michael Hudson-Doyle <email address hidden>
kill stupid config accessors
2012-09-26 Michael Hudson-Doyle <email address hidden>
return config objects that support access by attributes
2012-09-25 Michael Hudson-Doyle <email address hidden>
sort schema, mark a couple of types
2012-09-25 Michael Hudson-Doyle <email address hidden>
kill name arg to get_config
2012-09-25 Michael Hudson-Doyle <email address hidden>
mostly mechanically create schema for devices and the overall config
2012-09-24 Andy Doan <email address hidden>
post release bump
2012-09-24 Andy Doan <email address hidden>
release-0.18
2012.09
2012-09-21 Michael Hudson-Doyle <email address hidden>
do not explode if /etc/resolv.conf is missing in the testrootfs
2012-09-20 Michael Hudson-Doyle <email address hidden>
cheap hack to not die if /etc/resolv.conf is not present when setting up chroot partition
2012-09-21 Michael Hudson-Doyle <email address hidden>
remove never-used features that claimed to support testing ubuntu and android images with custom kernels
2012-09-07 Michael Hudson-Doyle <email address hidden>
kill equivalent feature for android
2012-09-07 Michael Hudson-Doyle <email address hidden>
brutally slay code and docs (!) to do with kernel matrix feature
2012-09-19 Antonio Terceiro <email address hidden>
Use qemu-system-arm by default
People who are trying architectures other than ARM can still override that in
their configuration if needed
