[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [Pulp-dev] pulp 3 PR testing with pulp-smash

On Thu, Feb 22, 2018 at 5:21 PM, Daniel Alley <dalley redhat com> wrote:
Would it be possible to have the required tests be Pulp core only, but to have an expanded set of non-mandatory smash tests which includes pulp_file?

Which would mean, the pulp_file smash test results would be there as a visual indicator, but wouldn't cause problems over the next few months before the plugin API is fully stabilized.

Yes we can. We would need to set it up as two separate Jenkins jobs. I believe GitHub allows setting a subset of checks as gating.


On Thu, Feb 22, 2018 at 4:57 PM, Dennis Kliban <dkliban redhat com> wrote:
tl;dr: which set of pulp-smash tests should run against pulpcore PRs? pulpcore + pulp_file or just pulpcore?

Jeremy and I are working to enable a new check for PRs against Pulp's 3.0-dev branch. This is going to be a jenkins job that installs pulpcore from the PR and then runs pulp-smash smoke tests against it. 

The smoke tests include both pulpcore and pulp_file tests. When testing PRs for pulp repository, should pulp_file also be installed thus allowing pulp-smash all the tests? The other option is to not install pulp_file and allow only the pulpcore tests to run.

If both pulpcore and pulp_file tests are required to pass to merge a PR, then we can get into a situation where the plugin API is intentionally changing and the tests can't pass until the change is introduced to pulp_file also. In such situations we could require the pulpcore-plugin package to have it's version bumped, which would mark the build as passing.

If only pulpcore tests are run we could get into a situation where the PR breaks the plugin API and we don't learn this until after the code is merged.

Which set of tests should run for pulpcore PRs?

Pulp-dev mailing list
Pulp-dev redhat com

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]