This is what I learned about building systems, not just scripts.

The Problem

I was manually checking LinkedIn and government job portals every week for Singapore IT roles. 20-30 minutes each time. Repetitive, soul-crushing, easily forgotten.

There had to be a better way.

The Solution

Built a scraper that does it automatically.

Real results:

• 1,939 IT jobs in a single run

• 8 minutes end-to-end

• $0.13 per execution

• 100% success rate (no silent failures)

• Zero code changes needed for different keywords or locations

That last point was the real win. I didn't want fragile code that breaks every time I change a search parameter. Just fill in the form and it works.

Why This Matters

Most people ask: "Why spend 4 days building something that takes 20 minutes to do manually?"

They're missing the point entirely.

It's not about the 20 minutes saved. It's about never having to think about it again. About brain space freed up for real problems. About knowing it runs every Sunday without any intervention.

Someone in the automation community said it perfectly: "I'd do it again even if the time saved never catches up to the time I spent building it."

That's the actual win.

The Technical Stack

I chose Apify + Playwright + Node.js deliberately — not for hype, but for reliability.

Architecture highlights:

• Modular handlers for each site (LinkedIn, MyCareersFuture, etc.)

• Single orchestrator that iterates sites × keywords × pages

• Graceful error handling (no silent failures, no mystery hangs)

• Cloud-deployed (not local, not fragile)

• Proxy rotation handled by Apify

Why modular from day 1?

I knew I'd want to add more sites eventually. The whole point was proving you could build an extensible scraper — not a one-off hack.

Adding a new site takes ~50 lines of code. Same pattern every time. That's the difference between a tool and a platform.

What Actually Took Time

Not the scraping logic. That's maybe 30% of the work.

The other 70%? Debugging. Getting selectors right when websites change. Handling edge cases. Testing different proxy configurations. Making sure it doesn't fail silently on empty pages or network timeouts.

That's what separates a script from a production system.

The Business Side

I published this on Apify Marketplace. Free tier (1 run/month), unlimited runs for paid users.

Not because I'm expecting to get rich. But because I believe in charging for reliability. And because giving something away for free feels incomplete.

Open source the code, charge for the hosted service. Both have their place.

What I'd Do Differently

1. Start with output format first. I spent too much time on architecture before I knew what the output should actually look like. Lesson learned.

2. Test with production proxies earlier. Found proxy rotation issues late in development. Would've caught them sooner in production-like conditions.

3. Document as you go. Retroactive documentation is brutal. Do it live.

What I Got Right

1. Modular from day 1. Not retrofitting structure later saved hours of refactoring.

2. Actual error handling. Not just try/catch and pray. Real graceful fallbacks for every failure mode I could think of.

3. Proof before polish. Got 1,939 jobs working before I touched the README. Proof comes first.

For Anyone Building Automation

A few hard-won thoughts:

You don't need AI agents for everything. Simple, reliable deterministic code beats fancy tech every single time. Make the business money first. Fancy comes later.

Build for reliability over speed. A scraper that runs 100 times and fails once is worse than one that runs 10 times perfectly.

Open source your code, charge for the service. The code is the trust signal. The hosting/reliability is the value.

The time investment doesn't need to "pay for itself." If your brain gets freedom, it already paid for itself.

The Result

GitHub (full source code): Multi-Site Jobs Scraper

Try it (no coding needed): Try on Apify Marketplace

Free tier available. I'm collecting feedback, not revenue.

Final Thought

Building something you'd actually use is wildly underrated. It forces you to solve real problems, not theoretical ones.

Start there.