Repository navigation
Broken links in Search engines #643
Description
Describe the bug
Merged or closed contributions results in broken links in Google searches.
To Reproduce
Steps to reproduce the behavior:
- Go to google.com
- Search for a keyword that could be part of a contribution
- Scroll down to the dzcode link provided by Google's search engine
- Click on the link and you get
A broken link?error (.eg: https://www.dzcode.io/contribute/manage-memory-in-a-better-way-github-2922582687)
Expected behavior
Closed contributions links should be removed from search engines as well
Screenshots
Environment:
- OS: MacOS
- Browser : Safari
- Version 18.4
Activity
ZibanPirate commentedon Apr 12, 2025
They will eventually be removed because such pages return a 404 code, but the problem is that Google doesn’t remove links immediately.
I can’t think of an ideal solution, but perhaps as a workaround, we can still serve these pages so users can still see the content, but also keep returning a 404 (not a 200) so Google will know they need to be deleted. Eventually, they will be deleted.
However, when do we clean up these pages after? How do we know they’re removed from Google? Should we rely on time, like deleting them after two weeks of closure?
omdxp commentedon Apr 12, 2025
We need to have a mechanism (probably another cron job per week) that do the search in famous search engines and see if they're still shown in the results; if they're not we can delete them