{"id":71435,"date":"2023-02-22T09:00:32","date_gmt":"2023-02-22T09:00:32","guid":{"rendered":"https:\/\/www.cryptocabaret.com\/?p=71435"},"modified":"2023-02-22T09:00:32","modified_gmt":"2023-02-22T09:00:32","slug":"how-i-do-automated-accessibility-testing-for-my-website","status":"publish","type":"post","link":"https:\/\/www.cryptocabaret.com\/?p=71435","title":{"rendered":"How I do automated accessibility testing for my website"},"content":{"rendered":"<p><span class=\"field field--name-title field--type-string field--label-hidden\">How I do automated accessibility testing for my website<\/span><br \/>\n<span class=\"field field--name-uid field--type-entity-reference field--label-hidden\"><a title=\"View user profile.\" href=\"https:\/\/opensource.com\/users\/dmundra\" class=\"username\">dmundra<\/a><\/span><br \/>\n<span class=\"field field--name-created field--type-created field--label-hidden\">Wed, 02\/22\/2023 &#8211; 03:00<\/span><\/p>\n<div class=\"clearfix text-formatted field field--name-body field--type-text-with-summary field--label-hidden field__item\">\n<p>This article covers adding accessibility tests to your site using <a href=\"https:\/\/pa11y.org\/\" target=\"_blank\" rel=\"noopener\">Pa11y<\/a> (<a href=\"https:\/\/github.com\/pa11y\/pa11y-ci\" target=\"_blank\" rel=\"noopener\">pa11y-ci<\/a> with <a href=\"https:\/\/www.deque.com\/axe\/\" target=\"_blank\" rel=\"noopener\">axe<\/a>) and <a href=\"https:\/\/www.cypress.io\/\" target=\"_blank\" rel=\"noopener\">Cypress<\/a> (with <a href=\"https:\/\/www.npmjs.com\/package\/cypress-axe\" target=\"_blank\" rel=\"noopener\">cypress-axe<\/a>) in <a href=\"https:\/\/docs.gitlab.com\/ee\/ci\/\" target=\"_blank\" rel=\"noopener\">GitLab CI\/CD<\/a>. I use a <a href=\"https:\/\/opensource.com\/article\/21\/9\/build-website-jekyll\" target=\"_blank\" rel=\"noopener\">Jekyll<\/a> website as an example, but any website technology that runs in CI\/CD can leverage this setup.<\/p>\n<h2>Prep your website<\/h2>\n<p>In addition to getting your website to run in CI\/CD, I recommend enabling an XML sitemap feature. A sitemap allows the accessibility tests to parse all URLs to find accessibility issues across the site. I recommend the <a href=\"https:\/\/github.com\/jekyll\/jekyll-sitemap\" target=\"_blank\" rel=\"noopener\">jekyll-sitemap plugin<\/a> for Jekyll sites.<\/p>\n<p>Collecting a list of all major URLs is a good alternate step if a sitemap is not possible. The URLs should cover all potential layouts of the website, such as pages with the highest traffic or the most landings. This approach won&#8217;t catch all accessibility issues, especially content level concerns, but it will test the layout and main pages.<\/p>\n<p>This scenario requires the <a href=\"https:\/\/www.npmjs.com\/\" target=\"_blank\" rel=\"noopener\">npm<\/a> or <a href=\"https:\/\/yarnpkg.com\/\" target=\"_blank\" rel=\"noopener\">yarn<\/a> package managers. I used <code>npm<\/code> for this article. If your project doesn&#8217;t have <code>npm<\/code> initialized, run the <code>npm init<\/code> command to create the <code>package.json<\/code> file.<\/p>\n<h2>Begin with Pa11y<\/h2>\n<p>Pa11y is a free and open source software that tests websites for accessibility issues. Pa11y-ci is the command line utility geared towards continuous integration (CI). Install pa11y-ci as a development dependency with <code>npm<\/code>:<\/p>\n<pre>\n<code class=\"language-shell\">$ npm i --save-dev pa11y-ci<\/code><\/pre>\n<p>After you complete the installation, edit the <code>package.json<\/code> and add the following commands to the <strong>scripts<\/strong> section:<\/p>\n<pre>\n<code class=\"language-json\">\"start-detached\": \"bundle exec jekyll serve --detach\",\n\"pa11y-ci:home\": \"pa11y-ci http:\/\/127.0.0.1:4000\",\n\"pa11y-ci:sitemap\": \"pa11y-ci --sitemap http:\/\/127.0.0.1:4000\/sitemap.xml --sitemap-find https:\/\/accessibility.civicactions.com --sitemap-replace http:\/\/127.0.0.1:4000 --sitemap-exclude \"\/*.pdf\"\"<\/code><\/pre>\n<ul>\n<li>start-detached: Starts the web server that will run Jekyll for testing.<\/li>\n<li>pa11y-ci:home: Runs pa11y-ci tests on the home page. Useful for troubleshooting.<\/li>\n<li>pa11y-ci:sitemap: Runs pa11y-ci tests using the sitemap and excludes PDFs. The sitemap will refer to the live site URLs, so replace those with local URLs for testing in the CI pipeline.<\/li>\n<\/ul>\n<p>Add a JSON file named <code>.pa11yci<\/code> that configures pa11y-ci with <a href=\"https:\/\/github.com\/pa11y\/pa11y#configuration\" target=\"_blank\" rel=\"noopener\">various options<\/a>. Here is a sample file:<\/p>\n<pre>\n<code class=\"language-json\">{\n  \"defaults\": {\n    \"concurrency\": 1,\n    \"standard\": \"WCAG2AA\",\n    \"runners\": [\"axe\", \"htmlcs\"],\n    \"ignore\": [\n      \"color-contrast\",\n      \"frame-tested\"\n    ],\n    \"chromeLaunchConfig\": {\n      \"args\": [\"--disable-dev-shm-usage\", \"--no-sandbox\", \"--disable-gpu\"]\n    },\n    \"reporters\": [\n      \"cli\",\n      [\".\/pa11y-reporter-junit.js\", { \"fileName\": \".\/pa11y-report-junit.xml\" }]\n    ]\n  }\n}<\/code><\/pre>\n<ul>\n<li>concurrency: I reduced this set to 1 because increasing it caused errors (<a href=\"https:\/\/github.com\/pa11y\/pa11y-ci\/issues\/168\" target=\"_blank\" rel=\"noopener\">https:\/\/github.com\/pa11y\/pa11y-ci\/issues\/168<\/a> covers the bug, which might be fixed).<\/li>\n<li>standard: I have stuck with the default WCAG2AA as the goal for this site.<\/li>\n<li>runners: I ran axe (run tests using axe-core) and htmlcs (default, run tests using HTML CodeSniffer) to cover all potential accessibility issues.<\/li>\n<li>ignore: With newer versions of axe and some of the changes to the site, I ran into color contrast false positives. I also have an embedded iframe that requires separate testing that axe will report about. I have follow-up issues to examine the axe results, so I am ignoring those criteria for now.<\/li>\n<li>chromeLaunchConfig: pa11y-ci uses Chrome, and I found that the GitLab CI pipeline requires that the Chrome browser runs properly in the pipeline.<\/li>\n<li>reports: I use the default command line reporter, but I also added a custom reporter that reports on the pa11y-ci results in a junit format. This came in handy for reporting the results in the GitLab CI pipeline.<\/li>\n<\/ul>\n<p>That&#8217;s it. Run this setup locally using <code>npm<\/code>, and you will see the following output (truncated for brevity):<\/p>\n<pre>\n<code class=\"language-shell\">dmundra in ~\/workspace\/accessibility\/accessibility on branch main &gt; npm run start-detached\n\n&gt; start-detached\n&gt; bundle exec jekyll serve --detach\n\nConfiguration file: \/Users\/dmundra\/workspace\/accessibility\/accessibility\/_config.yml\n            Source: \/Users\/dmundra\/workspace\/accessibility\/accessibility\n       Destination: \/Users\/dmundra\/workspace\/accessibility\/accessibility\/_site\n Incremental build: disabled. Enable with --incremental\n      Generating...\n                    done in 8.217 seconds.\n Auto-regeneration: disabled when running server detached.\n    Server address: http:\/\/127.0.0.1:4000\nServer detached with pid '14850'. Run `pkill -f jekyll' or `kill -9 14850' to stop the server.\ndmundra in ~\/workspace\/accessibility\/accessibility on branch main &gt; npm run pa11y-ci:sitemap\n\n&gt; pa11y-ci:sitemap\n&gt; pa11y-ci --sitemap http:\/\/localhost:4000\/sitemap.xml --sitemap-exclude \"\/*.pdf\"\n\nRunning Pa11y on 110 URLs:\n &gt; http:\/\/localhost:4000\/guide\/glossary - 0 errors\n &gt; http:\/\/localhost:4000\/guide\/introduction - 0 errors\n &gt; http:\/\/localhost:4000\/guide\/history - 0 errors\n &gt; http:\/\/localhost:4000\/guide\/design - 0 errors\n...\n\n&#x2714; 110\/110 URLs passed<\/code><\/pre>\n<p>The site passes the tests. <a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\/-\/jobs\/3630213194\" target=\"_blank\" rel=\"noopener\">Here<\/a> is an example job running in GitLab. The pa11y configuration continues to test all site pages for accessibility issues and report on them.<\/p>\n<p>What does an error look like? Here is an example:<\/p>\n<pre>\n<code class=\"language-shell\"> &gt; http:\/\/localhost:4000\/guide\/introduction - 1 errors\n\nErrors in http:\/\/localhost:4000\/guide\/introduction:\n\n \u2022 <ul> and <\/ul><ol> must only directly contain <li>, <\/li><\/ol><\/code><\/pre>\n<p>You get a count of the number of errors at a given URL and then details on the accessibility issue. It also displays a link to the criteria being violated and the location in the HTML of the issue.<\/p>\n<h2>Try Cypress<\/h2>\n<p>Cypress is a JavaScript testing framework and is very helpful in writing tests that interact with the site and assert that features work as expected. The setup for Cypress is very similar to pa11y-ci in terms of installation with <code>npm<\/code>.<\/p>\n<pre>\n<code class=\"language-shell\">$ npm i --save-dev cypress cypress-axe cypress-real-events<\/code><\/pre>\n<p>After the installation is complete, edit the <code>package.json<\/code> and add the following commands to the <strong>scripts<\/strong> section:<\/p>\n<pre>\n<code class=\"language-json\">\"cypress-tests\": \"cypress run --browser chrome --headless\"<\/code><\/pre>\n<ul>\n<li>cypress-tests: Run the Cypress tests with a headless Chrome browser.<\/li>\n<\/ul>\n<p>When launching Cypress for the first time, you get a wizard to create the <a href=\"https:\/\/docs.cypress.io\/guides\/references\/configuration\" target=\"_blank\" rel=\"noopener\">configuration file<\/a>. Here is a sample file:<\/p>\n<pre>\n<code class=\"language-json\">const { defineConfig } = require('cypress')\n\nmodule.exports = defineConfig({\n  video: true,\n  videosFolder: 'cypress\/results',\n  reporter: 'junit',\n  reporterOptions: {\n    mochaFile: 'cypress\/results\/junit.[hash].xml',\n    toConsole: false,\n  },\n  screenshotsFolder: 'cypress\/results\/screenshots',\n  e2e: {\n    \/\/ We've imported your old cypress plugins here.\n    \/\/ You may want to clean this up later by importing these.\n    setupNodeEvents(on, config) {\n      return require('.\/cypress\/plugins\/index.js')(on, config)\n    },\n    baseUrl: 'http:\/\/localhost:4000',\n  },\n})<\/code><\/pre>\n<ul>\n<li>video: Take videos of the tests, which are helpful for troubleshooting.<\/li>\n<li>videosFolder: Defines the video storage folder.<\/li>\n<li>reporter: Set to junit to make it easier to report the results in the GitLab CI pipeline.<\/li>\n<li>reporterOptions: Includes a path for the junit files and the keyword [hash] to preserve unique reports for each test file (otherwise, the file is overwritten). Skip the console output for the reporter and use the default output.<\/li>\n<li>screenshotsFolder: Defines the screenshot storage folder (useful for troubleshooting).<\/li>\n<li>e2e: References the local URL of the site and the plugins.<\/li>\n<\/ul>\n<p>After setting up Cypress and writing some tests (see below for examples), run the tests locally using <code>npm<\/code>. You will see the following output (truncated for brevity):<\/p>\n<pre>\n<code class=\"language-shell\">dmundra in ~\/workspace\/accessibility\/accessibility on branch main &gt; npm run cypress-tests\n\n&gt; cypress-tests\n&gt; cypress run --browser chrome --headless\n\n====================================================================================================\n\n  (Run Starting)\n\n  \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n  \u2502 Cypress:        11.2.0                                                                         \u2502\n  \u2502 Browser:        Chrome 109 (headless)                                                          \u2502\n  \u2502 Node Version:   v18.10.0 (\/usr\/local\/Cellar\/node\/18.10.0\/bin\/node)                             \u2502\n  \u2502 Specs:          5 found (accordion.cy.js, home.cy.js, images.cy.js, menu.cy.js, search.cy.js)  \u2502\n  \u2502 Searched:       cypress\/e2e\/**\/*.cy.{js,jsx,ts,tsx}                                            \u2502\n  \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n                                                                                                    \n  Running:  search.cy.js                                                                    (5 of 5)\n\n  (Results)\n\n  \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n  \u2502 Tests:        1                                                                                \u2502\n  \u2502 Passing:      1                                                                                \u2502\n  \u2502 Failing:      0                                                                                \u2502\n  \u2502 Pending:      0                                                                                \u2502\n  \u2502 Skipped:      0                                                                                \u2502\n  \u2502 Screenshots:  0                                                                                \u2502\n  \u2502 Video:        true                                                                             \u2502\n  \u2502 Duration:     2 seconds                                                                        \u2502\n  \u2502 Spec Ran:     search.cy.js                                                                     \u2502\n  \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n\n\n  (Video)\n\n  -  Started processing:  Compressing to 32 CRF                                                     \n  -  Finished processing: \/Users\/dmundra\/workspace\/accessibility\/accessibility\/cypres    (0 seconds)\n                          s\/results\/search.cy.js.mp4     \n \n...\n  (Run Finished)\n\n\n       Spec                                              Tests  Passing  Failing  Pending  Skipped  \n  \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n  \u2502 &#x2714;  search.cy.js                             00:02        1        1        -        -        - \u2502\n\n...<\/code><\/pre>\n<p>While Pa11y-ci can test interactivity, Cypress and its plugins can do much more. For a Jekyll site, I found that pa11y-ci did not catch any accessibility issues in mobile drop-down menu, dynamic search, or accordion features. I ran Cypress tests to interact with the elements (like performing searches, clicking menus, or clicking the accordion) and then checked if the results still passed accessibility tests. Here is the <a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\/-\/blob\/143f875853\/cypress\/e2e\/search.cy.js\" target=\"_blank\" rel=\"noopener\">search example<\/a>:<\/p>\n<pre>\n<code class=\"language-json\">describe('Search', () =&gt; {\n  it('should be accessible', () =&gt; {\n    cy.visit('\/search')\n    cy.get('#search-input').type('accessibility')\n    cy.checkA11yWithMultipleViewPorts()\n  })\n})<\/code><\/pre>\n<p>Here is a quick <a href=\"https:\/\/drive.google.com\/file\/d\/1aHQuVkeBoEyla4ASwWL6BBxJ-JGYJTHa\/view?usp=sharing\" target=\"_blank\" rel=\"noopener\">video<\/a> of the running test.<\/p>\n<p>The above test visits the search page, types the word &#8220;accessibility&#8221; in the search field, and then checks the results for accessibility issues. I use the <a href=\"https:\/\/www.npmjs.com\/package\/cypress-axe\" target=\"_blank\" rel=\"noopener\">cypress-axe<\/a> plugin to check accessibility issues with axe core, just like pa11y-ci. I have wrapped the cypress-axe functions in a function to test multiple window sizes and report on the issues in a table format.<\/p>\n<p>I also use the plugin <a href=\"https:\/\/github.com\/dmtrKovalenko\/cypress-real-events\" target=\"_blank\" rel=\"noopener\">cypress-real-events<\/a> to interact with the site with a keyboard to check that the features are keyboard-accessible. Keyboard accessibility is a critical consideration (<em>Operable<\/em> principle of WCAG), and having an automated test that can confirm the features are keyboard accessible means that, maybe, there is one less test to run manually. You can see an example of the test <a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\/-\/blob\/143f875853\/cypress\/e2e\/menu.cy.js#L27-41\" target=\"_blank\" rel=\"noopener\">here<\/a>.<\/p>\n<p>Here is an example of what an error looks like:<\/p>\n<pre>\n<code class=\"language-shell\">  Running:  a11y\/anonymous_a11y.cy.js                                                      (1 of 36)\ncy.log(): Accessibility scanning: Home (\/)\ncy.log(): 4 accessibility violations were detected\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n\u2502 (index) \u2502           id           \u2502   impact   \u2502                                    description                                     \u2502 nodes \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502    0    \u2502      'image-alt'       \u2502 'critical' \u2502   'Ensures <img> elements have alternate text or a role of none or presentation'   \u2502   4   \u2502\n\u2502    1    \u2502      'link-name'       \u2502 'serious'  \u2502                       'Ensures links have discernible text'                        \u2502   3   \u2502\n\u2502    2    \u2502 'page-has-heading-one' \u2502 'moderate' \u2502 'Ensure that the page, or at least one of its frames contains a level-one heading' \u2502   1   \u2502\n\u2502    3    \u2502        'region'        \u2502 'moderate' \u2502                'Ensures all page content is contained by landmarks'                \u2502   2   \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518<\/code><\/pre>\n<p>Cypress logs provide a count of the number of errors at a given URL and then details on what the accessibility issue is, the impact, and its location.<\/p>\n<p>You can find additional details and examples in the <a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\/-\/tree\/143f875853\/cypress\" target=\"_blank\" rel=\"noopener\">Cypress folder<\/a>.<\/p>\n<h2>Use the GitLab CI\/CD<\/h2>\n<p>Now that you have pa11y-ci and Cypress running locally, see how to run automated accessibility tests in GitLab using <a href=\"https:\/\/docs.gitlab.com\/ee\/ci\/\" target=\"_blank\" rel=\"noopener\">CI\/CD features<\/a>. The GitLab repository is available <a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\" target=\"_blank\" rel=\"noopener\">here<\/a>. Here is the <code>.gitlab-ci.yml<\/code> file setup:<\/p>\n<pre>\n<code class=\"language-shell\">stages:\n- test\n\ncache:\n  key: ${CI_COMMIT_REF_SLUG}\n  paths:\n    - node_modules\/\n    - .npm\/\n    - vendor\/ruby\n\ndefault:\n  image: ruby:2\n  before_script:\n    - apt-get update\n    - apt-get install -yq gconf-service libasound2 libatk1.0-0 libc6 libcairo2 libcups2 libdbus-1-3 libexpat1 libfontconfig1 libgcc1 libgconf-2-4 libgdk-pixbuf2.0-0 libglib2.0-0 libgtk-3-0 libnspr4 libpango-1.0-0 libpangocairo-1.0-0 libstdc++6 libx11-6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libxss1 libxtst6 ca-certificates fonts-liberation libnss3 lsb-release xdg-utils wget libgbm1 xvfb\n    - apt-get install -y nodejs npm\n    - bundle install -j $(nproc) --path vendor\/ruby\n    - npm ci --cache .npm --prefer-offline\n    - npm run start-detached\n\npa11y-tests:\n  stage: test\n  script:\n    - npm run pa11y-ci:sitemap\n  artifacts:\n    when: always\n    reports:\n      junit:\n        - pa11y-report-junit.xml\n    expire_in: 1 day\n\ncypress-tests:\n  stage: test\n  script:\n    # Install chrome browser manually, taken from https:\/\/github.com\/cypress-io\/cypress-docker-images\/blob\/master\/browsers\/node16.14.2-slim-chrome100-ff99-edge\/Dockerfile#L48\n    - wget --no-verbose -O \/usr\/src\/google-chrome-stable_current_amd64.deb \"http:\/\/dl.google.com\/linux\/chrome\/deb\/pool\/main\/g\/google-chrome-stable\/google-chrome-stable_105.0.5195.125-1_amd64.deb\"\n    - dpkg -i \/usr\/src\/google-chrome-stable_current_amd64.deb\n    - rm -f \/usr\/src\/google-chrome-stable_current_amd64.deb\n    - npm run cypress-tests\n  artifacts:\n    when: always\n    paths:\n      - cypress\/results\/\n    reports:\n      junit:\n        - cypress\/results\/*.xml\n    expire_in: 1 day<\/code><\/pre>\n<p>The file currently defines only one stage <strong>test<\/strong> and caches folders that store dependencies when installed. Then:<\/p>\n<ol>\n<li>Steps used by all stages:\n<ol>\n<li>Use the Ruby version 2 image because it is compatible with the current Jekyll installation.<\/li>\n<li>I install many dependencies based on the documentation at <a href=\"https:\/\/github.com\/puppeteer\/puppeteer\/blob\/main\/docs\/troubleshooting.md#running-puppeteer-on-gitlabci\" target=\"_blank\" rel=\"noopener\">running puppeteer<\/a> on GitLab. Install <code>node<\/code> and <code>npm<\/code> to install site dependencies.<\/li>\n<li>Install the Jekyll Ruby dependencies.<\/li>\n<li>Install the Cypress and pa11y-ci dependencies via <code>npm<\/code>.<\/li>\n<li>Start the web server.<\/li>\n<\/ol>\n<\/li>\n<li>Run the pa11y-ci to test the site and capture the output to a file.*<\/li>\n<li>Install the Chrome browser dependencies in cypress-tests using steps provided by Cypress in their Docker image configurations. Run the Cypress tests and capture the output to files.*<\/li>\n<\/ol>\n<p>* Capture the output of Cypress and pa11y-ci tests as junit XML files.<\/p>\n<p>Here is an example screenshot of the GitLab pipeline (taken from\u00a0<a href=\"https:\/\/gitlab.com\/civicactions\/accessibility\/-\/pipelines\/744894072\" target=\"_blank\" rel=\"noopener\">https:\/\/gitlab.com\/civicactions\/accessibility\/-\/pipelines\/744894072<\/a>):<\/p>\n<article class=\"align-center media media--type-image media--view-mode-default\">\n<div class=\"field field--name-field-media-image field--type-image field--label-hidden field__item\">  <img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/www.cryptocabaret.com\/wp-content\/uploads\/2023\/02\/1screenshot.png\" width=\"910\" height=\"1214\" alt=\"GitLab pipeline\"><\/div>\n<div class=\"field field--name-field-caption field--type-text-long field--label-hidden caption field__item\"><span class=\"caption__byline\">Image by: <\/span><\/p>\n<p>(Daniel Mundra, CC BY-SA 4.0)<\/p>\n<\/div>\n<\/article>\n<p>Here is an example of the test results in the same pipeline:<\/p>\n<article class=\"align-center media media--type-image media--view-mode-default\">\n<div class=\"field field--name-field-media-image field--type-image field--label-hidden field__item\">  <img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/www.cryptocabaret.com\/wp-content\/uploads\/2023\/02\/2summary.png\" width=\"936\" height=\"250\" alt=\"Test results\"><\/div>\n<div class=\"field field--name-field-caption field--type-text-long field--label-hidden caption field__item\"><span class=\"caption__byline\">Image by: <\/span><\/p>\n<p>(Daniel Mundra, CC BY-SA 4.0)<\/p>\n<\/div>\n<\/article>\n<div class=\"embedded-resource-list callout-float-right\">\n<div class=\"field field--name-title field--type-string field--label-hidden field__item\">Our favorite resources about open source<\/div>\n<div class=\"field field--name-links field--type-link field--label-hidden field__items\">\n<div class=\"field__item\"><a href=\"https:\/\/opensource.com\/downloads\/cheat-sheet-git?intcmp=7016000000127cYAAQ\">Git cheat sheet<\/a><\/div>\n<div class=\"field__item\"><a href=\"https:\/\/developers.redhat.com\/cheat-sheets\/advanced-linux-commands\/?intcmp=7016000000127cYAAQ\">Advanced Linux commands cheat sheet<\/a><\/div>\n<div class=\"field__item\"><a href=\"https:\/\/opensource.com\/tags\/alternatives?intcmp=7016000000127cYAAQ\">Open source alternatives<\/a><\/div>\n<div class=\"field__item\"><a href=\"https:\/\/www.redhat.com\/en\/services\/training\/rh024-red-hat-linux-technical-overview?intcmp=7016000000127cYAAQ\">Free online course: RHEL technical overview<\/a><\/div>\n<div class=\"field__item\"><a href=\"https:\/\/opensource.com\/downloads\/cheat-sheets?intcmp=7016000000127cYAAQ\">Check out more cheat sheets<\/a><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<p><a href=\"https:\/\/docs.gitlab.com\/ee\/ci\/testing\/unit_test_reports.html\" target=\"_blank\" rel=\"noopener\">GitLab CI\/CD automatically take junit XML files<\/a> and outputs them in a clear format. Cypress tests provide the junit XML output as part of their features (see above). I created a custom reporter for pa11y-ci to output the format in <a href=\"https:\/\/github.com\/CivicActions\/accessibility\/blob\/143f875853\/pa11y-reporter-junit.js\" target=\"_blank\" rel=\"noopener\">junit<\/a> (credit to <a href=\"https:\/\/github.com\/macieklewkowicz\/pa11y-reporter-junit\" target=\"_blank\" rel=\"noopener\">macieklewkowicz\/pa11y-reporter-junit<\/a>).<\/p>\n<p>Note: GitLab version 12.8+ supports Pa11y accessibility tests (see <a href=\"https:\/\/docs.gitlab.com\/ee\/ci\/testing\/accessibility_testing.html\" target=\"_blank\" rel=\"noopener\">https:\/\/docs.gitlab.com\/ee\/ci\/testing\/accessibility_testing.html<\/a> for details). The above setup allows for customization of the pa11y-ci and also targeting of local URLs. I recommend using their options for live sites.<\/p>\n<h2>Wrap up<\/h2>\n<p>Using the above steps, you can provide accessibility testing for your site locally and in CI. This process helps you track and fix accessibility issues on your site and in the content. An important caveat about automated testing is that it only catches <a href=\"https:\/\/www.deque.com\/blog\/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues\/\" target=\"_blank\" rel=\"noopener\">57% of issues<\/a>, so you definitely want to include manual testing with your accessibility testing.<\/p>\n<h3>Further reading and examples<\/h3>\n<ul>\n<li><a href=\"https:\/\/accessibility.civicactions.com\/posts\/automated-accessibility-testing-leveraging-github-actions-and-pa11y-ci-with-axe\" target=\"_blank\" rel=\"noopener\">Automated accessibility testing: Leveraging GitHub Actions and pa11y-ci with axe<\/a>.<\/li>\n<li><a href=\"https:\/\/andrewmee.com\/posts\/automated-accessibility-testing-node-travis-ci-pa11y\/\" target=\"_blank\" rel=\"noopener\">Automated accessibility testing with Travis CI<\/a>.<\/li>\n<li><a href=\"https:\/\/medium.com\/@f3igao\/how-to-automate-web-accessibility-testing-921512bdd4bf\" target=\"_blank\" rel=\"noopener\">How to Automate Web Accessibility Testing<\/a>.<\/li>\n<li><a href=\"https:\/\/timdeschryver.dev\/blog\/setting-up-cypress-with-axe-for-accessibility\" target=\"_blank\" rel=\"noopener\">Setting up Cypress with axe for accessibility &#8211; Tim Deschryver<\/a>.<\/li>\n<li><a href=\"https:\/\/docs.cypress.io\/guides\/continuous-integration\/gitlab-ci\" target=\"_blank\" rel=\"noopener\">GitLab CI | Cypress Documentation<\/a>.<\/li>\n<\/ul>\n<p>Thank you to Marissa Fox and Mike Gifford for your support, thoughts, and feedback.<\/p>\n<\/div>\n<div class=\"clearfix text-formatted field field--name-field-article-subhead field--type-text-long field--label-hidden field__item\">\n<p>Follow along with this example of performing accessibility tests in GitLab with Pa11y and Cypress on a Jekyll website.<\/p>\n<\/div>\n<div class=\"field field--name-field-lead-image field--type-entity-reference field--label-hidden field__item\">\n<article class=\"media media--type-image media--view-mode-caption\">\n<div class=\"field field--name-field-media-image field--type-image field--label-hidden field__item\">  <img decoding=\"async\" loading=\"lazy\" src=\"https:\/\/www.cryptocabaret.com\/wp-content\/uploads\/2023\/02\/browser_web_internet_website.png\" width=\"1040\" height=\"584\" alt=\"Digital creative of a browser on the internet\" title=\"Digital creative of a browser on the internet\"><\/div>\n<\/article>\n<\/div>\n<div class=\"field field--name-field-tags field--type-entity-reference field--label-hidden field__items\">\n<div class=\"field__item\"><a href=\"https:\/\/opensource.com\/tags\/accessibility\" hreflang=\"en\">Accessibility<\/a><\/div>\n<div class=\"field__item\"><a href=\"https:\/\/opensource.com\/tags\/gitlab\" hreflang=\"en\">GitLab<\/a><\/div>\n<\/p><\/div>\n<div class=\"hidden field field--name-field-listicle-title field--type-string field--label-hidden field__item\">What to read next<\/div>\n<div class=\"field field--name-field-default-license field--type-list-string field--label-hidden field__item\"><a rel=\"license\" href=\"http:\/\/creativecommons.org\/licenses\/by-sa\/4.0\/\"><br \/>\n        <img decoding=\"async\" alt=\"Creative Commons License\" src=\"https:\/\/www.cryptocabaret.com\/wp-content\/uploads\/2023\/02\/cc-by-sa--33.png\" title=\"This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.\"><\/a>This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License.<\/div>\n<section class=\"field field--name-field-comments field--type-comment field--label-hidden comment-wrapper\">\n<div class=\"comments__count\">\n<div class=\"login\"><a href=\"https:\/\/opensource.com\/user\/register?absolute=1\">Register<\/a> or <a href=\"https:\/\/opensource.com\/user\/login?destination=\/feed&amp;absolute=1\">Login<\/a> to post a comment.<\/div>\n<\/p><\/div>\n<\/section>\n<p class=\"wpematico_credit\"><small>Powered by <a href=\"http:\/\/www.wpematico.com\" target=\"_blank\" rel=\"noopener\">WPeMatico<\/a><\/small><\/p>\n","protected":false},"excerpt":{"rendered":"<p>How I do automated accessibility testing for my website dmundra Wed, 02\/22\/2023 &#8211; 03:00 This article covers adding accessibility tests to your site using Pa11y (pa11y-ci with axe) and Cypress (with cypress-axe) in GitLab CI\/CD. I use a Jekyll website as an example, but any website technology that runs in CI\/CD can leverage this setup. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":71436,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[307],"tags":[],"class_list":["post-71435","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-open-source"],"_links":{"self":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts\/71435","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=71435"}],"version-history":[{"count":0,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/posts\/71435\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=\/wp\/v2\/media\/71436"}],"wp:attachment":[{"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=71435"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=71435"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.cryptocabaret.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=71435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}