Hey guys, decided to run some updates of modules via the WebUI last night and definitely running into some major issues now. Web UI went down, rebooted and now it’s painfully broken. Web UI came back up, but Asterisk is down, and starting asterisk manually (fwconsole restart) doesn’t work unfortunately, citing corrupt firewall rules. I tried updating via SSH, (sudo yum upgrade) and now it’s just endlessly spitting out “error: rpmdb: damaged header #591 retrieved – skipping.”, consuming 100% resources. Been running for ages. Forced it to reboot and we’re back to square one. Short of grabbing my backup from a month ago, any ideas? I think it’s about time I virtualise this, snapshots would be real handy right now.
Edit: Just logged back in and now it’s saying I’m out of disk space. Sigh, time to grab the backup.
As I was reading through this, I was thinking ‘sounds like he’s out of disk space’…
You can probably fix it manually, without needing to do a restore. The first thing to do is add some more disk space, or, find out where all your existing disk space is gone, and fix that up.
After you fix the disk space issue, a ‘yum update’ and ‘fwconsole ma downloadinstall’ should fix everything else.
Thanks for the tip. I just upgraded my storage (clonezilla to the rescue) and will give them a go now. All I had to do was clone and then fix the swap partition (re-create partition after main partition resize, new UUID in fstab). /tmp directory was pretty huge (2.5GB). Mind you, was running off an old 8GB SD card (one of those test things that I never had a chance to change), so I was probably playing with fire there.
Anyway, just ran “yum update” and it looks like it was doing it’s thing, then it started spamming the console with “error: rpmdb: damaged header #591 retrieved – skipping” once again. sigh. Any ideas?
Update, I ran a script to rebuilt the RPM database from here and things seem to be semi-working for now, firewall seems broken but I’ll tackle that after hours.