Is running a PHPUnit coverage report with `--strict-coverage` worth it? I’ve been doing this on a personal project, and every time I make some foundational change within Symfony (like something to do with the DI container or kernel, I have to update every single one of my test classes so it knows it’s okay to use the new class.
Maybe there’s a better way? I’ve only just begun doing this recently, so I’m new to it.
PHPUnit: рабочий нерабочий covers
Зачем нужны covers метки? Почему их так хочется сломать? Как это сделать? Рассказ будет извилист, но идея проста: улучшить навигацию, не задев покрытие.
Write your tests using #PHPUnit 12.x and run them using PHPUnit 7.5 - 12.x to verify cross-version compatibility.
Thanks to @jrf_nl and contributors!
1.x → supports PHPUnit 4.8–9.x
2.x → supports PHPUnit 5.7–10.x
3.x → supports PHPUnit 6.4–11.x (**)
4.x → supports PHPUnit 7.5–12.x (**)
https://github.com/yoast/phpunit-polyfills
https://github.com/sponsors/jrfnl
(**: except PHPUnit 10 for running tests)
If someone is running PHPUnit in a docker container and wonders why they can't filter by namespaces....
I finally managed it with
phpunit --filter 'My\\\\\\\Namespace'
Why 7? I have no idea! As long as it works! BTW: The quotes are important! Without them it'll be - - you guessed it correctly - 14 backslashes... 🙈
Code-Coverage with PCOV in a mono-repo
Today I finally managed to find and fix an issue that we had for some time with Code-Coverage generation with PCOV in github actions.
To give you a bit of background, imagine a project that uses 2 separate folders where one contains the business-logic and one contains the framework-related code. The framework-related part contains a lot of integration and end-to-end test that – of course – also use and therefore test the business-logic.
Every PullRequest runs the tests both from the bunsiness-logic part as well as from the framework-related part. And to give the developers feedback how well the changed code is covered with tests we collect coverage data and generate a patch-coverage report. I wrote about that some time back. So we now get a comment each in the PR that contains the untested lines in the business-logic related part and the framework-related part respectively.
That worked flawlessly for the business-logic related part as those were mostly unit-tests that do not rely upon the framework-related code.
The framework-related part though didn’t work that great. It always showed the business-logic related parts as untested even though I knew they were tested.
So… Why?
After some digging I realized that there is a difference in how the coverage is created between Xdebug and pcov – yes! There is obviously a difference as those are different tools. But there is also a difference in what is included in the reports.
And today I figured out what it was and why it broke our proces in our specific setup in GitHub actions.
For a start: We have our code in two folders: business and framework.
We also have a framework/phpunit.xml file that contains this code:
Ben Ramsey
Habr
github.com/ghostwriter
Alerta! Alerta!
andreas.heigl.org<?xml version="1.0" encoding="UTF-8"?><phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/12.1/phpunit.xsd"
processIsolation="false"
stopOnFailure="false"
>
<coverage/>
<source>
<include>
<directory>src</directory>
<directory>../business/src</directory>
</include>
</source>
</phpunit>So coverage shall be collected from the current src folder as well as from the business/src folder.
This works as expected when you run phpunit with Xdebug as coverage-generator.
With pcov as coverage-generator it worked locally as expected.
But not in our GitHub Actions…
Enter pcov.directory
After a bit of digging I learned that there is the pcov.directory ini-setting that determines which code is considered for coverage. So I tested my GitHub Actions by adding some test-code like this:
jobs:
tests:
steps:
- name: "Checkout"
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
with:
ref: ${{ github.event.pull_request.head.ref }}
- name: "Install PHP"
uses: shivammathur/setup-php@44454db4f0199b8b9685a5d763dc37cbf79108e1 # v2
with:
extensions: "intl, pdo_mysql, zip, imap, xml, soap, apcu, redis"
php-version: "8.4"
coverage: "pcov"
- name: "check pcov.directory"
run: |
php -i | grep pcovRunning this in GitHub Actions provided me with output similar to this:
Run php -i | grep pcov
/etc/php/8.4/cli/conf.d/20-pcov.ini,
pcov
pcov.directory => /home/runner/work/<org-name>/<repo-name>
pcov.exclude => none
pcov.initial.memory => 65336 bytes
pcov.initial.files => 64/home/runner/work/<org-name>/<repo-name> was exactly what i expected! So why the heck does it seemingly not work?
After some more testing I realized that I was at one point doing a cd framework before running PHPUnit.
- name: "Run tests"
run: |
cd framework
vendor/phpunit/phpunit/phpunit --coverage-php /tmp/${{ github.sha }}_coverage.covShould that cause some issues?
So I added the php -i | grep pcov after the PHPUnit run like this:
- name: "Run tests"
run: |
cd framework
vendor/phpunit/phpunit/phpunit --coverage-php /tmp/${{ github.sha }}_coverage.cov
php -i | grep pcovAnd what was that? Suddenly I got
pcov.directory => /home/runner/work/<org-name>/<repo-name>/framework/src🤯
Suddenly the pcov directory was set to something totally different. And with that setting it was clear that I couldn’t get what I expected as now only content from framework/src/ would be considered as covered. And everything from the business folder was not considered for coverage.
So the only question left was whether that was something happening deep in PHPUnits logic or just based on changing the directory.
Long story short: It’s just based on the change directory. In essence setup-php sets up pcov so that the current folder is considered as pcov.directory. As pcov itself adds src when that folder is available that explains the changed folder.
But is there a way to change that? To use a fixed folder despite the current working directory being changed?
Yes! There is.
I added ini-values: "pcov.directory=$GITHUB_WORKSPACE" to the setup-php directive like this:
- name: "Install PHP"
uses: shivammathur/setup-php@44454db4f0199b8b9685a5d763dc37cbf79108e1 # v2
with:
extensions: "intl, pdo_mysql, zip, imap, xml, soap, apcu, redis"
ini-values: "pcov.directory=$GITHUB_WORKSPACE"
php-version: "8.4"
coverage: "pcov"And all of a sudden the pcov.directory stayed regardless of where I changed the working directory to.
And so with this little change I was again able to get the coverage-data from the business folder whenever i run tests in the framework folder.
Helpful links:
#coverage #pcov #php #phpunitI was about to request jokingly an assertRoughlyEquals() method in #phpunit to avoid errors like "Failed asserting that 5.656000000000001 is identical to 5.656" when testing with floats.
Then I realized it already exists ! 😂
https://docs.phpunit.de/en/12.5/assertions.html#assertequalswithdelta
Handy !
Freshly tested every two months: the most important #PHPUnit news:
https://phpunit.expert/articles/phpunit-update.html?ref=mastodon
The any() matcher for mock objects is deprecated in #PHPUnit 13.
But migration is easier than you think and leads to better tests:
https://phpunit.expert/articles/from-anti-pattern-to-clarity.html?ref=mastodon
The any() matcher for mock objects is deprecated in #PHPUnit 13.
But migration is easier than you think and leads to better tests:
https://phpunit.expert/articles/from-anti-pattern-to-clarity.html?ref=mastodon
Dark mode in HTML coverage reports for PHPUnit (version 13) is beautiful!
Thank you, @sebastian!
Halibut 🐟
Sebastian Bergmann 
