Database Normalization vs. Denormalization: When to Use Each?

4. File URL Being Overwritten by Status Polling

Problem:

• The callback was correctly setting file_url in the database
• But the /video-podcast/status endpoint was being called every 3 seconds by frontend
• The status endpoint was overwriting the file_url with whatever the microservice returned (often None)
• This caused the file_url to disappear from the database

Solution:

• Modified the status endpoint to only update file_url if it's not already set:

  if microservice_status.get("status") == "completed" and not video_job.file_url:
      video_job.file_url = microservice_status.get("file_url")
  elif microservice_status.get("status") == "completed" and video_job.file_url:
      print("Video podcast already has file_url, not overwriting")

The main issue was the file_url overwrite problem (#4), which was causing the video URLs to disappear from the database even though the callback was working correctly. The fix ensures that once the callback sets the file_url, the status polling won't overwrite it.

  updated_at = db.Column(db.DateTime, server_default=func.now(), onupdate=func.now())

  if microservice_status.get("status") == "completed" and not video_job.file_url:
      video_job.file_url = microservice_status.get("file_url")
  elif microservice_status.get("status") == "completed" and video_job.file_url:
      print("Video podcast already has file_url, not overwriting")

User

this is interesting problem i faced: File URL Being Overwritten by Status Polling

write a linkedin post on this

Cursor

Absolutely! Here’s a LinkedIn post you can use to share your experience and learning:

🐛 Debugging a Sneaky Backend Bug: When Polling Overwrites Your Data

Today I solved a really interesting backend bug that I think many engineers can relate to—especially if you’re building async workflows with polling and callbacks.

The Problem

We have a video generation feature where:

• A microservice generates a video and calls our /video-callback endpoint with the final video URL.
• The frontend polls /video-podcast/status every few seconds to update the UI with progress.

Everything looked fine in the logs:
The callback was received, the video URL was saved to the database, and the logs confirmed it.

But when I checked the database manually, the file_url was missing!
It was as if the callback never happened.

The Culprit: Polling Overwrites

After a deep dive, I realized the frontend’s polling endpoint was:

• Fetching the latest status from the microservice every 3 seconds.
• Unconditionally updating the file_url in the database with whatever the microservice returned (which was often null or empty until the job was fully processed).

So, every time the frontend polled, it would overwrite the correct file_url set by the callback with a blank value!

The Fix

I updated the status endpoint to only set the file_url if it wasn’t already set:

if microservice_status.get(\"status\") == \"completed\" and not video_job.file_url:
    video_job.file_url = microservice_status.get(\"file_url\")

Now, once the callback sets the URL, polling can’t overwrite it.

Lessons Learned

• Be careful with polling endpoints that update state!
  They should never blindly overwrite important fields.
• Callbacks and polling can race each other.
  Always design for idempotency and data integrity.
• Logs can be misleading if you don’t consider all code paths.
  Double-check what’s actually happening in the DB.

Have you ever run into a similar bug with polling or async workflows? How did you solve it?
Let’s connect and share war stories! 🚀

#backend #python #flask #bugfix #async #polling #microservices #softwareengineering #devlife

Feel free to personalize or shorten as you like!

if microservice_status.get(\"status\") == \"completed\" and not video_job.file_url:
    video_job.file_url = microservice_status.get(\"file_url\")