------------------------------------------- 1) vxdist list (disks weren't broken, we could talk to them) 2) vxprint -htrL (too much information) only needed: vxprint 3) vxplex -o force -g xxxxarchive dis raidvol-02 4) vxvol start raidvol 5) fsck /xxxxarchive 6) mount /xxxxarchive >--> fsck /xxxxarchive ** /dev/vx/rdsk/xxxxarchive/raidvol ** Last Mounted on /xxxxarchive ** Phase 1 - Check Blocks and Sizes ** Phase 2 - Check Pathnames ** Phase 3 - Check Connectivity ** Phase 4 - Check Reference Counts ** Phase 5 - Check Cyl groups 504298 files, 15099504 used, 18962517 free (14613 frags, 2368488 blocks, 0.0% fragmentation) commands: -------------- vxplex -g xxxxarchive att raidvol raidvol-01 (already attached) didn't need to run... vxdist list pkginfo -l VRTSvxvm (ver 3.04) vxprint -htrL vxplex -g xxxxarchive att raidvol raidvol-a vxvol -l vxvol startall vxvol start raidvol vxrecover vxdg list vxvol -g xxxxarc start raidvol vxmend -g xxxxarchive fix unstale xxxxarc01-01 vxmend -g xxxxarchive fix unstale xxxxarc02-01 vxmend -g xxxxarchive fix unstale xxxxarc03-01 vxmend -g xxxxarchive fix unstale xxxxarc04-01 >--> vxmend -g xxxxarchive fix unstale xxxxarc01-01 vxvm:vxmend: ERROR: Subdisk xxxxarc01-01 is not stale, detached, kdetached or writeonly; nothing to fix ----------------------------------------------------- root@xxxxsr [/] >--> vxplex -g xxxxarchive dis raidvol-02 vxvm:vxplex: ERROR: Volume raidvol: dissociate would leave volume with no valid Raid5 logs; operation requires -o force vxplex -o force -g xxxxarchive dis raidvol-02 vxprint v raidvol raid5 DETACHED 69206016 - REPLAY - - pl raidvol-01 raidvol ENABLED 69209856 - ACTIVE - - sd xxxxarc01-01 raidvol-01 ENABLED 34604928 0 - - - sd xxxxarc02-01 raidvol-01 ENABLED 34604928 0 - - - sd xxxxarc03-01 raidvol-01 ENABLED 34604928 0 - - - ----------------------------------------------------- root@xxxxsr [/] >--> vxplex -o force -g xxxxarchive det raidvol-02 vxvm:vxplex: ERROR: Plex raidvol-02 is not associated ----------------------------------------------------- root@xxxxsr [/] >--> vxvol start raidvol total 1110 -rw------- 1 root other 367176 Apr 17 14:58 core drwxr-xr-x 2 root other 512 Oct 5 2000 xxxxarchive -rw-r--r-- 1 root other 831 Apr 16 01:05 errors-vx-xxxxarchive -rw-r--r-- 1 root other 1563 Apr 17 15:07 fixraid -rw-r--r-- 1 root other 3100 Apr 17 14:23 vxprint-htrL.txt -rw-r--r-- 1 root other 7348 Apr 17 14:04 vxprint-htrl.txt ========================================================================= -rw-r--r-- 1 root other 3100 Apr 17 14:23 vxprint-htrL.txt Disk group: rootdg DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE dg rootdg default default 0 970772413.1025.xxxxsr dm rootdisk c0t0d0s2 sliced 4711 35363560 - v opt fsgen ENABLED ACTIVE 4099440 ROUND - pl opt-01 opt ENABLED ACTIVE 4099440 CONCAT - RW sd rootdisk-04 opt-01 rootdisk 4052319 4099440 0 c0t0d0 ENA v rootvol root ENABLED ACTIVE 513608 ROUND - pl rootvol-01 rootvol ENABLED ACTIVE 513608 CONCAT - RW sd rootdisk-02 rootvol-01 rootdisk 2101551 513608 0 c0t0d0 ENA v swapvol swap ENABLED ACTIVE 2101552 ROUND - pl swapvol-01 swapvol ENABLED ACTIVE 2101552 CONCAT - RW sd rootdisk-B0 swapvol-01 rootdisk 35363559 1 0 c0t0d0 ENA sd rootdisk-01 swapvol-01 rootdisk 0 2101551 1 c0t0d0 ENA v usr fsgen ENABLED ACTIVE 3072224 ROUND - pl usr-01 usr ENABLED ACTIVE 3072224 CONCAT - RW sd rootdisk-03 usr-01 rootdisk 8151759 3072224 0 c0t0d0 ENA v var fsgen ENABLED ACTIVE 1437160 ROUND - pl var-01 var ENABLED ACTIVE 1437160 CONCAT - RW sd rootdisk-05 var-01 rootdisk 2615159 1437160 0 c0t0d0 ENA Disk group: xxxxarchive DG NAME NCONFIG NLOG MINORS GROUP-ID DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE V NAME USETYPE KSTATE STATE LENGTH READPOL PREFPLEX PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE dg xxxxarchive default default 30000 970777210.1101.xxxxsr dm xxxxarc01 c1t0d0s2 sliced 4711 35363560 - dm xxxxarc02 c1t1d0s2 sliced 4711 35363560 - dm xxxxarc03 c2t8d0s2 sliced 4711 35363560 - dm xxxxarc04 c2t9d0s2 sliced 4711 35363560 - v raidvol raid5 DETACHED REPLAY 69206016 RAID - pl raidvol-01 raidvol ENABLED ACTIVE 69209856 RAID 3/32 RW sd xxxxarc01-01 raidvol-01 xxxxarc01 0 34604928 0/0 c1t0d0 ENA sd xxxxarc02-01 raidvol-01 xxxxarc02 0 34604928 1/0 c1t1d0 ENA sd xxxxarc03-01 raidvol-01 xxxxarc03 0 34604928 2/0 c2t8d0 ENA pl raidvol-02 raidvol ENABLED LOG 4712 CONCAT - RW sd xxxxarc04-01 raidvol-02 xxxxarc04 0 4712 0 c2t9d0 ENA ========================================================================= -rw-r--r-- 1 root other 7348 Apr 17 14:04 vxprint-htrl.txt Disk group: rootdg Group: rootdg info: dgid=970772413.1025.xxxxsr version: 60 copies: nconfig=default nlog=default minors: >= 0 Disk: rootdisk info: diskid=970772414.1032.xxxxsr assoc: device=c0t0d0s2 type=sliced flags: autoconfig device: pubpath=/dev/vx/dmp/c0t0d0s7 privpath=/dev/vx/dmp/c0t0d0s4 devinfo: publen=35363560 privlen=4711 Volume: opt info: len=4099440 type: usetype=fsgen state: state=ACTIVE kernel=ENABLED cdsrecovery=0/0 (clean) assoc: plexes=opt-01 policies: read=ROUND exceptions=GEN_DET_SPARSE flags: open writeback logging: type=REGION loglen=0 serial=0/0 (disabled) apprecov: seqno=0 recov_id=0 device: minor=6 bdev=153/6 cdev=153/6 path=/dev/vx/dsk/rootdg/opt perms: user=root group=root mode=0600 Plex: opt-01 info: len=4099440 type: layout=CONCAT state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=opt sd=rootdisk-04 flags: busy complete Subdisk: rootdisk-04 info: disk=rootdisk offset=4052319 len=4099440 assoc: vol=opt plex=opt-01 (offset=0) flags: enabled busy device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 Volume: rootvol info: len=513608 type: usetype=root state: state=ACTIVE kernel=ENABLED cdsrecovery=0/0 (clean) assoc: plexes=rootvol-01 policies: read=ROUND exceptions=GEN_DET_SPARSE flags: open writeback logging: type=NONE apprecov: seqno=0 recov_id=0 device: minor=0 bdev=153/0 cdev=153/0 path=/dev/vx/dsk/rootdg/rootvol perms: user=root group=root mode=0600 Plex: rootvol-01 info: len=513608 type: layout=CONCAT state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=rootvol sd=rootdisk-02 flags: busy complete Subdisk: rootdisk-02 info: disk=rootdisk offset=2101551 len=513608 assoc: vol=rootvol plex=rootvol-01 (offset=0) flags: enabled busy device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 Volume: swapvol info: len=2101552 type: usetype=swap state: state=ACTIVE kernel=ENABLED cdsrecovery=0/0 (clean) assoc: plexes=swapvol-01 policies: read=ROUND exceptions=GEN_DET_SPARSE flags: open writeback logging: type=REGION loglen=0 serial=0/0 (disabled) apprecov: seqno=0 recov_id=0 device: minor=8 bdev=153/8 cdev=153/8 path=/dev/vx/dsk/rootdg/swapvol perms: user=root group=root mode=0600 Plex: swapvol-01 info: len=2101552 type: layout=CONCAT state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=swapvol sd=rootdisk-B0,rootdisk-01 flags: busy complete Subdisk: rootdisk-B0 info: disk=rootdisk offset=35363559 len=1 assoc: vol=swapvol plex=swapvol-01 (offset=0) flags: enabled busy comment: Remap of block 0 device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 utils: p0=Block0 Subdisk: rootdisk-01 info: disk=rootdisk offset=0 len=2101551 assoc: vol=swapvol plex=swapvol-01 (offset=1) flags: enabled busy device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 Volume: usr info: len=3072224 type: usetype=fsgen state: state=ACTIVE kernel=ENABLED cdsrecovery=0/0 (clean) assoc: plexes=usr-01 policies: read=ROUND exceptions=GEN_DET_SPARSE flags: open writeback logging: type=NONE apprecov: seqno=0 recov_id=0 device: minor=7 bdev=153/7 cdev=153/7 path=/dev/vx/dsk/rootdg/usr perms: user=root group=root mode=0600 Plex: usr-01 info: len=3072224 type: layout=CONCAT state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=usr sd=rootdisk-03 flags: busy complete Subdisk: rootdisk-03 info: disk=rootdisk offset=8151759 len=3072224 assoc: vol=usr plex=usr-01 (offset=0) flags: enabled busy device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 Volume: var info: len=1437160 type: usetype=fsgen state: state=ACTIVE kernel=ENABLED cdsrecovery=0/0 (clean) assoc: plexes=var-01 policies: read=ROUND exceptions=GEN_DET_SPARSE flags: open writeback logging: type=REGION loglen=0 serial=0/0 (disabled) apprecov: seqno=0 recov_id=0 device: minor=5 bdev=153/5 cdev=153/5 path=/dev/vx/dsk/rootdg/var perms: user=root group=root mode=0600 Plex: var-01 info: len=1437160 type: layout=CONCAT state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=var sd=rootdisk-05 flags: busy complete Subdisk: rootdisk-05 info: disk=rootdisk offset=2615159 len=1437160 assoc: vol=var plex=var-01 (offset=0) flags: enabled busy device: device=c0t0d0s2 path=/dev/vx/dmp/c0t0d0s7 diskdev=152/7 Disk group: xxxxarchive Group: xxxxarchive info: dgid=970777210.1101.xxxxsr version: 60 copies: nconfig=default nlog=default minors: >= 30000 Disk: xxxxarc01 info: diskid=970777209.1098.xxxxsr assoc: device=c1t0d0s2 type=sliced flags: autoconfig device: pubpath=/dev/vx/dmp/c1t0d0s4 privpath=/dev/vx/dmp/c1t0d0s3 devinfo: publen=35363560 privlen=4711 Disk: xxxxarc02 info: diskid=970777945.1103.xxxxsr assoc: device=c1t1d0s2 type=sliced flags: autoconfig device: pubpath=/dev/vx/dmp/c1t1d0s4 privpath=/dev/vx/dmp/c1t1d0s3 devinfo: publen=35363560 privlen=4711 Disk: xxxxarc03 info: diskid=970779368.1105.xxxxsr assoc: device=c2t8d0s2 type=sliced flags: autoconfig device: pubpath=/dev/vx/dmp/c2t8d0s4 privpath=/dev/vx/dmp/c2t8d0s3 devinfo: publen=35363560 privlen=4711 Disk: xxxxarc04 info: diskid=970780375.1107.xxxxsr assoc: device=c2t9d0s2 type=sliced flags: autoconfig device: pubpath=/dev/vx/dmp/c2t9d0s4 privpath=/dev/vx/dmp/c2t9d0s3 devinfo: publen=35363560 privlen=4711 Volume: raidvol info: len=69206016 type: usetype=raid5 state: state=REPLAY kernel=DETACHED cdsrecovery=34603008/0 (clean) assoc: plexes=raidvol-01,raidvol-02 policies: read=RAID exceptions=NO_OP flags: closed recover-parity (done) writecopy writeback logging: type=RAID5 loglen=2880 serial=0/0 (enabled) apprecov: seqno=0 recov_id=0 device: minor=30000 bdev=153/30000 cdev=153/30000 path=/dev/vx/dsk/xxxxarchive/raidvol perms: user=root group=root mode=0600 Plex: raidvol-01 info: len=69209856 type: layout=RAID columns=3 width=32 state: state=ACTIVE kernel=ENABLED io=read-write assoc: vol=raidvol sd=xxxxarc01-01,xxxxarc02-01,xxxxarc03-01 flags: complete Subdisk: xxxxarc01-01 info: disk=xxxxarc01 offset=0 len=34604928 assoc: vol=raidvol plex=raidvol-01 (column=0 offset=0) flags: enabled device: device=c1t0d0s2 path=/dev/vx/dmp/c1t0d0s4 diskdev=152/12 Subdisk: xxxxarc02-01 info: disk=xxxxarc02 offset=0 len=34604928 assoc: vol=raidvol plex=raidvol-01 (column=1 offset=0) flags: enabled device: device=c1t1d0s2 path=/dev/vx/dmp/c1t1d0s4 diskdev=152/20 Subdisk: xxxxarc03-01 info: disk=xxxxarc03 offset=0 len=34604928 assoc: vol=raidvol plex=raidvol-01 (column=2 offset=0) flags: enabled device: device=c2t8d0s2 path=/dev/vx/dmp/c2t8d0s4 diskdev=152/28 Plex: raidvol-02 info: len=4712 type: layout=CONCAT state: state=LOG kernel=ENABLED io=read-write assoc: vol=raidvol sd=xxxxarc04-01 flags: raidlog complete Subdisk: xxxxarc04-01 info: disk=xxxxarc04 offset=0 len=4712 assoc: vol=raidvol plex=raidvol-02 (offset=0) flags: enabled device: device=c2t9d0s2 path=/dev/vx/dmp/c2t9d0s4 diskdev=152/36 >--> umount /xxxxarchive umount: warning: /xxxxarchive not in mnttab umount: /xxxxarchive not mounted ----------------------------------------------------- root@xxxxsr [/] >--> mount /xxxxarchive mount: /dev/vx/dsk/xxxxarchive/raidvol is already mounted, /xxxxarchive is busy, or the allowable number of mount points has been exceeded ----------------------------------------------------- root@xxxxsr [/] ORIGINAL ERROR: >--> fsck -F ufs -y /dev/vx/dsk/xxxxarchive/raidvol ** /dev/vx/dsk/xxxxarchive/raidvol CANNOT READ: BLK 16 CONTINUE? yes THE FOLLOWING SECTORS COULD NOT BE READ: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 CANNOT READ: BLK 16 CONTINUE? yes THE FOLLOWING SECTORS COULD NOT BE READ: 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 ----------------------------------------------------- root@xxxxsr [/]