Mikael Lirbank•16mo ago

Hey, we're running Playwright in a

Hey, we're running Playwright in a GitHub action after Vercel has deployed (the GH Action is triggered by Vercel when a Vercel deploy is successful). This works great as the Playwright tests are running on the app deployed at Vercel. But for a lot of tests we seed the Convex DB with a test user just before we run the Playwright test. We use ConvexHttpClient() for seeding the DB, and to be able to use ConvexHttpClient we need the NEXT_PUBLIC_CONVEX_URL - which is dynamic as we use Convex Preview deploys. Any ideas for how to provide GitHub Actions with the current NEXT_PUBLIC_CONVEX_URL? And I guess I prolly need some secret too, like CONVEX_DEPLOY_KEY to be able to write to the DB with ConvexHttpClient...

6 Replies

Mikael LirbankOP•16mo ago

I wonder if I can pass variables from Vercel post deploy to GitHub actions, reading here https://docs.github.com/en/rest/deployments?apiVersion=2022-11-28#list-deployment-statuses

GitHub Docs

Deployments - GitHub Docs

Mikael LirbankOP•16mo ago

Here is the GH Action:

# https://vercel.com/guides/how-can-i-run-end-to-end-tests-after-my-vercel-preview-deployment
# https://playwright.dev/docs/ci-intro
# https://github.com/vercel/next.js/tree/canary/examples/with-playwright

name: Playwright Tests
on:
  deployment_status:
jobs:
  test:
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success'
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - name: Log environment_url
        run: echo "${{ github.event.deployment_status.environment_url }}"
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18
      - uses: pnpm/action-setup@v2
        with:
          version: 8
      - name: Install dependencies
        run: pnpm install
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test
        env:
          BASE_URL: ${{ github.event.deployment_status.environment_url }}
      - uses: actions/upload-artifact@v3
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

# https://vercel.com/guides/how-can-i-run-end-to-end-tests-after-my-vercel-preview-deployment
# https://playwright.dev/docs/ci-intro
# https://github.com/vercel/next.js/tree/canary/examples/with-playwright

name: Playwright Tests
on:
  deployment_status:
jobs:
  test:
    if: github.event_name == 'deployment_status' && github.event.deployment_status.state == 'success'
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - name: Log environment_url
        run: echo "${{ github.event.deployment_status.environment_url }}"
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
        with:
          node-version: 18
      - uses: pnpm/action-setup@v2
        with:
          version: 8
      - name: Install dependencies
        run: pnpm install
      - name: Install Playwright Browsers
        run: npx playwright install --with-deps
      - name: Run Playwright tests
        run: npx playwright test
        env:
          BASE_URL: ${{ github.event.deployment_status.environment_url }}
      - uses: actions/upload-artifact@v3
        if: always()
        with:
          name: playwright-report
          path: playwright-report/
          retention-days: 30

Here I am setting the BASE_URL env from the github.event.deployment_status.environment_url that I belive is provided by Vercel via a webhook call to GH... I wonder if I can customize what Vercel send to GH Actions...

Mikael LirbankOP•16mo ago

I guess I could create a webhook in the Vercel UI, but it seems involved for this. Especially since a webhook is already sent to GH Actions on deploy success.

Mikael LirbankOP•16mo ago

I think Vercel actually create a GH deployment status, not a webhook (I think 🙃) https://docs.github.com/en/rest/deployments/statuses?apiVersion=2022-11-28#create-a-deployment-status

GitHub Docs

Deployment statuses - GitHub Docs

sshader•16mo ago

Hmm so you have a playwright test that creates a test user using the ConvexHttpClient (and then presumably logs in as that user and does things)? To brainstorm a bit (I'm not super familiar with playwright): One option is to do all your seed data setup as part of your vercel deploy (something like npx convex deploy --cmd 'npm run build' --preview-run seed). This would mean all your tests rely on the same seed data, which might not fit your needs. Another option (which is kind of a hack, but might work) would be to stick your NEXT_PUBLIC_CONVEX_URL in the UI somewhere selectable by playwright? And I think you can gate this so that it only happens in preview environments and not prod. This should let you call public convex functions using the HTTP client. Calling an internal function from playwright might be trickier since (like you said) we need to get something like CONVEX_DEPLOY_KEY for each Convex preview deployment. Let me think on that for a bit...

Cilla•16mo ago

@sshader We 're on the same track as you - we publish the NEXT_PUBLIC_CONVEX_URL in the UI and feed the tests with the value. Not super elegant, but works! (I'll check out if I can remove it from prod - very helpful if so!) Thanks a bunch!

Hey, we're running Playwright in a

Did you find this page helpful?