{"id":412,"date":"2017-11-13T21:50:10","date_gmt":"2017-11-13T20:50:10","guid":{"rendered":"https:\/\/ilmarkerm.eu\/blog\/?p=412"},"modified":"2017-11-13T21:53:37","modified_gmt":"2017-11-13T20:53:37","slug":"regular-database-restore-tests-are-important","status":"publish","type":"post","link":"https:\/\/ilmarkerm.eu\/blog\/2017\/11\/regular-database-restore-tests-are-important\/","title":{"rendered":"Regular database restore tests are important"},"content":{"rendered":"<p>I came to work today morning and there was an alert in my inbox saying that one of the large databases failed the nightly restore test.<br \/>\nLooking into RMAN logs I saw that recovery failed when applying archivelogs and error was something I have never seen before:<\/p>\n<pre><code>ORA-00756: recovery detected a lost write of a data block\nORA-10567: Redo is inconsistent with data block (file# 3, block# 37166793, file offset is 3350347776 bytes)\nORA-10564: tablespace SYSAUX\nORA-01110: data file 3: '\/nfs\/...'\nORA-10560: block type 'FIRST LEVEL BITMAP BLOCK'\n<\/code><\/pre>\n<p>Version: 12.1.0.2 with 2017-08 bundle patch<\/p>\n<p>Looking at MOS I see two bugs that could match:<\/p>\n<ul>\n<li>Bug 22302666  ORA-753 ORA-756 or ORA-600 [3020] with KCOX_FUTURE after RMAN Restore \/ PITR with BCT after Open Resetlogs in 12c<\/li>\n<\/ul>\n<p>but this bugfix is already included in 2017-08 bundle patch, and:<\/p>\n<ul>\n<li>Bug 23589471  ORA-600 [3020] KCOX_FUTURE or ORA-756 Lost Write detected during Recovery of a RMAN backup that used BCT<\/li>\n<\/ul>\n<p>Looks like this matches quite well with my situation and the note has a really scary sentence in it: <strong>Incremental RMAN backups using BCT may miss to backup some blocks.<\/strong><\/p>\n<p>Bugs exists in modern software, bugs exist even in rock solid Oracle database backup and recovery procedures. It doesn&#8217;t matter if the backup was completed successfully, the state of the backup can only be determened when a restore is attempted. So please start testing your backups. Regularly. Daily.<\/p>\n<p><a href=\"\/blog\/2017\/11\/about-database-backup-and-recovery\/\">Non-technical post about database backup and recovery<\/a><br \/>\n<a href=\"\/blog\/oracle-imagecopy-backup\/\">Scripts to implement incremental-forever backup strategy in Oracle and testing backups automatically<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>I came to work today morning and there was an alert in my inbox saying that one of the large databases failed the nightly restore test. Looking into RMAN logs I saw that recovery failed when applying archivelogs and error was something I have never seen before: ORA-00756: recovery detected a lost write of a [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-412","post","type-post","status-publish","format-standard","hentry","category-blog-entry"],"_links":{"self":[{"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/posts\/412","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/comments?post=412"}],"version-history":[{"count":4,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/posts\/412\/revisions"}],"predecessor-version":[{"id":416,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/posts\/412\/revisions\/416"}],"wp:attachment":[{"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/media?parent=412"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/categories?post=412"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ilmarkerm.eu\/blog\/wp-json\/wp\/v2\/tags?post=412"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}