Time-dependent flaky tests: why your CI fails at midnight and passes at breakfast

The build that only fails on the last day of the month

Here's a pattern I've seen at three different companies, in three different languages. A test suite is green for weeks. Then on January 31st, four tests fail on main at 6:00 PM Pacific. Nobody touched that code. Someone reruns the job at 7:15, it passes, everyone shrugs and merges the release. On February 28th, the same four tests fail again.

Nobody connects the two incidents, because who correlates test failures by day of the month? The on-call engineer sees a red build, sees it's "one of those flaky ones," and hits rerun. The failure signature is real, reproducible, and screaming its root cause — and it gets erased by a retry button every single time.

Time-dependent tests are my favorite class of flaky test to hunt, because unlike race conditions they're not actually nondeterministic. They're perfectly deterministic functions of an input nobody wrote down: the wall clock. Your test doesn't fail randomly. It fails at 23:50 UTC, or during the DST transition, or when today + 14 days crosses a month boundary. It just looks random because your CI history doesn't have a column for "what time was it in Kiribati."

Time is a global variable you never declared

Every call to new Date(), Date.now(), time.Now(), datetime.now(), or System.currentTimeMillis() inside code under test is a read from mutable global state. We'd never accept a test that read an undeclared global config value and asserted on it. But we merge clock reads into business logic constantly, because time feels like a constant. It isn't. It's a global that mutates a thousand times per second and occasionally does genuinely weird things, like repeating an hour.

Here's the shape of the bug, in a test that will pass roughly 355 days a year:

test("trial expires 14 days after signup", () => {
  const trial = createTrial({ startedAt: new Date() });
  const expected = new Date();
  expected.setDate(expected.getDate() + 14);
  expect(trial.expiresAt.getDate()).toBe(expected.getDate());
});

If createTrial computes expiry by adding milliseconds and the test computes it with setDate, they'll agree — until a DST boundary sits inside the 14-day window and the two calculations drift by an hour, which flips the calendar date. Or until the two new Date() calls straddle midnight, which happens when your nightly cron kicks off at 23:59 UTC. The assertion is comparing two different clock reads and hoping they landed in the same day. Usually they do. Usually.

The five clock bugs behind most time flakiness

After enough of these hunts, the root causes sort into a short list:

• Midnight boundaries. Two clock reads on opposite sides of 00:00. Anything asserting "same day," "today's report," or date-bucketed grouping is exposed. These cluster hard around your nightly build schedule.
• Month and year rollovers. getDate() + 14, "first of next month" logic, fiscal-quarter math. These fail on a predictable handful of calendar days and pass everywhere else, which is why they survive for years.
• DST transitions. Code assumes every day has 24 hours or every timestamp maps to exactly one local time. In America/New_York, 2:30 AM on the spring-forward date doesn't exist, and on the fall-back date it exists twice. Two build failures a year, both mystifying.
• Runner timezone vs. laptop timezone. The test encodes an assumption about local time. Your laptop is in America/Los_Angeles; the CI runner is UTC. The test "only fails in CI," which everyone reads as infrastructure flakiness. It's not — it's a different value of the undeclared global.
• Elapsed-time assertions. expect(duration).toBeLessThan(100) passes on your M-series laptop and fails on a busy shared runner. This one isn't strictly a calendar bug, but it's the same sin: asserting on the wall clock instead of controlling it.

Notice that none of these are fixed by rerunning. The rerun passes because the clock moved, not because anything got better. You didn't resolve the failure — you waited it out.

The fix: inject the clock, then pick hostile times

The structural fix is the same in every language: treat the current time as an input, not an ambient fact. In Go, that means a clock interface instead of naked time.Now() calls:

type Clock interface {
    Now() time.Time
}

type realClock struct{}

func (realClock) Now() time.Time { return time.Now() }

// In tests:
type fixedClock struct{ t time.Time }

func (c fixedClock) Now() time.Time { return c.t }

In JavaScript you rarely need the abstraction because the test runner can hijack the global clock for you:

beforeEach(() => {
  jest.useFakeTimers();
  // Deliberately hostile: 10 minutes before midnight, on the day
  // clocks spring forward in the US, in a leap year.
  jest.setSystemTime(new Date("2028-03-12T23:50:00Z"));
});

afterEach(() => {
  jest.useRealTimers();
});

The second half of that snippet is the part teams skip, and it's the part that matters. Most engineers who freeze the clock freeze it to something comfortable — noon on a Tuesday in June. That makes the test deterministic, sure, but deterministic-and-blind. You've pinned the global to its friendliest possible value and declared victory.

Pin hostile times on purpose. Keep a small shared set of them — last day of the month, DST transitions in both directions, February 29, 23:59:30 UTC — and parameterize date-sensitive test suites across the set. When a suite fails under 2028-03-12T23:50:00Z and passes under noon-in-June, you've found a production bug, not a test bug. Users hit month boundaries too. Your tests were flaky precisely because your code was wrong for a few hours a month, and the flake was the only alarm going off.

Detection: hunt for calendar-clustered failures

Static hunting gets you the first wave. Grep the codebase for naked clock reads in code that tests exercise:

git grep -nE 'new Date\(\)|Date\.now\(\)|time\.Now\(\)|datetime\.(now|today)\(' -- ':!vendor' ':!node_modules'

You'll get noise — logging timestamps are fine — but every hit in domain logic is a candidate. The deeper problem is the flakes already in your suite, and for those the signal you need is temporal clustering: does this test's failure rate spike at particular hours of the day, days of the month, or twice a year in March and November? No individual engineer sees that pattern, because each person witnesses one failure and one green rerun. The pattern only exists in aggregate, across weeks of build history.

This is exactly the kind of test detection work that needs your CI history as a dataset rather than a scroll of red and green icons. When BuildPulse flags a flaky test, the timestamps of its failures are part of the evidence — and a test that fails almost exclusively between 23:00 and 01:00 UTC has essentially confessed. If you're triaging by hand today, even a spreadsheet of (test name, failure timestamp) pulled from your JUnit XML will surface the worst offenders in an afternoon.

You can also make CI adversarial about time instead of waiting for the calendar to come to you:

# .github/workflows/tz-canary.yml
name: Timezone canary
on:
  schedule:
    - cron: "45 23 * * *" # straddle UTC midnight on purpose
jobs:
  test:
    strategy:
      matrix:
        tz: ["UTC", "Pacific/Kiritimati", "Pacific/Midway", "America/New_York"]
    runs-on: ubuntu-latest
    env:
      TZ: ${{ matrix.tz }}
    steps:
      - uses: actions/checkout@v4
      - run: npm ci && npm test

Pacific/Kiritimati (UTC+14) and Pacific/Midway (UTC-11) put 25 hours between your extremes, so any "what day is it" disagreement between test and code gets flushed out daily instead of ambushing you quarterly. Run it as a scheduled canary, not a merge gate — you want the signal without adding four suite runs to every PR.

The rerun problem, if you have auditors

Here's the part that matters if you're a SOC2 or ISO 27001 shop, or anywhere CI gates are part of your change-management story. A passing test suite before merge is a control. When a suite fails and someone reruns it until it's green, the evidence attached to that change now says: the control failed, we tried again with no changes, and then it passed. For a genuinely environmental blip, fine. For a test that deterministically fails near month-end, you've got a recurring control failure with no investigation record — and reruns are the mechanism actively destroying the record.

The defensible workflow is boring and better: when a test is identified as flaky, quarantine it explicitly — tracked, ticketed, and excluded from the merge gate as a documented exception rather than a silent retry. Now your evidence says "known issue TEST-482, time-zone handling in billing expiry, quarantined on March 3, fix merged March 11." That's a story you can tell an auditor with a straight face. It's also just faster for the team, since nobody burns twenty minutes rerunning a job the calendar has already decided against. We've written about why retries paper over flaky tests rather than fix them, and the quarantine workflow docs cover how to make the exception trail automatic instead of tribal.

One more reason to care as a leader: time-dependent flakes are corrosive to test reliability out of proportion to their count. Because they cluster — four tests failing together at midnight, twice — they train engineers to distrust correlated failures, which are precisely the ones most likely to be real. The whole value of CI flakiness management is protecting the credibility of red. A test that fails on schedule and gets rerun on schedule is a standing lesson to your team that red means "try again," and that lesson is expensive to unteach.

Where to start Monday

• Grep for naked clock reads in domain logic; make time an injected input for anything doing date math.
• Freeze test clocks to hostile times — month-end, DST transitions, 23:59 UTC — not comfortable ones.
• Stand up a timezone canary job that runs across extreme TZ values on a schedule that straddles UTC midnight.
• Pull failure timestamps from your CI history and look for time-of-day and day-of-month clustering. Automate it if you can; spreadsheet it if you can't.
• Replace rerun-until-green with explicit, ticketed quarantine, so your change-management evidence records the exception instead of erasing it.

The clock is the one dependency every test in your suite shares and almost none of them declare. Declare it, control it, and one of your most confusing classes of CI flakiness turns into ordinary, fixable bugs — with the paper trail to prove it.