I'm posting the results of my format JFFS + 3 reboots 10+ minutes apart. This is the 4th time the JFFS has been formatted in the past 2 weeks since my 384.19 + ASUS JFFS resize
journey nightmare FUBAR started.
- I've had the J* scripts using the USB for a week +, with no CRC / JFFS errors being reported.
- This AM I unmount, then remove the USB, the, backup the JFFS (without that large file earlier), then select "Format JFFS" + APPLY, then reboot.
- Sit 10+ minutes, reboot
- Sit 10+ minutes, reboot
- Sit 10+ minutes, reboot
- Sit 10 minutes, reconnect the USB, restore the JFFS, reboot.
- Verified no CRC or GC errors being reported in router's logs.
- At 1007 I moved connmon from USB back to JFFS with connmon AMTM GUI toggle
- At 1013, the CRC errors / No space for GC errors returned.
Code:
Sep 7 10:11:00 spdMerlin: Starting speedtest using auto-selected server for WAN interface
Sep 7 10:11:02 kernel: jffs2: Data CRC dc07b7ed != calculated CRC 34d07a98 for node at 02308c38
..
Sep 7 10:13:53 kernel: jffs2: Data CRC dc07b7ed != calculated CRC 34d07a98 for node at 02308c38
Sep 7 10:13:53 kernel: jffs2: read_cache_page() returned error: -5
Sep 7 10:13:53 kernel: jffs2: Error garbage collecting node at 02308c38!
Sep 7 10:13:53 kernel: jffs2: No space for garbage collection. Aborting GC thread
Sep 7 10:13:55 kernel: jffs2: Data CRC dc07b7ed != calculated CRC 34d07a98 for node at 02308c38
Sep 7 10:13:55 kernel: jffs2: read_cache_page() returned error: -5
Sep 7 10:13:55 kernel: jffs2: Error garbage collecting node at 02308c38!
....
Moved code back to USB, rebooted the router. The CRC errors ceased but these 3 bad blocks remain in the NAND.
...
May 5 01:05:11 kernel: nand: Macronix NAND 256MiB 3,3V 8-bit
May 5 01:05:11 kernel: nand: 256 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
May 5 01:05:11 kernel: bcm63xx_nand ff801800.nand: Adjust timing_1 to 0x65324458 timing_2 to 0x80040e54
May 5 01:05:11 kernel: bcm63xx_nand ff801800.nand: detected 256MiB total, 128KiB blocks, 2KiB pages, 16B OOB, 8-bit, BCH-4
May 5 01:05:11 kernel: Bad block table found at page 131008, version 0x01
May 5 01:05:11 kernel: Bad block table found at page 130944, version 0x01
May 5 01:05:11 kernel: nand_read_bbt: bad block at 0x0000032e0000
May 5 01:05:11 kernel: nand_read_bbt: bad block at 0x000007b00000
May 5 01:05:11 kernel: nand_read_bbt: bad block at 0x00000f880000
May 5 01:05:11 kernel: >>>>> For primary mtd partition rootfs, cferam/vmlinux.lz mounted as JFFS2, vmlinux fs mounted as UBIFS <<<<<
May 5 01:05:11 kernel: Secondary mtd partition rootfs_update detected as JFFS2 for cferam/vmlinux source and UBIFS for vmlinux filesystem
May 5 01:05:11 kernel: setup_mtd_parts: misc indx 2 name misc3 nvram configured size 1
May 5 01:05:11 kernel: setup_mtd_parts: name misc3 configured size 0x00100000 offset 0xF600000
May 5 01:05:11 kernel: setup_mtd_parts: misc indx 1 name misc2 nvram configured size 47
May 5 01:05:11 kernel: setup_mtd_parts: name misc2 configured size 0x02f00000 offset 0xC700000
May 5 01:05:11 kernel: setup_mtd_parts: misc indx 0 name misc1 nvram configured size 8
May 5 01:05:11 kernel: setup_mtd_parts: name misc1 configured size 0x00800000 offset 0xBF00000
May 5 01:05:11 kernel: Creating 11 MTD partitions on "brcmnand.0":
....
No GC or CRC errors reported since.
df -hT
Filesystem Type Size Used Available Use% Mounted on
ubi:rootfs_ubifs ubifs 77.2M 60.6M 16.6M 79% /
devtmpfs devtmpfs 214.9M 0 214.9M 0% /dev
tmpfs tmpfs 215.0M 196.0K 214.8M 0% /var
tmpfs tmpfs 215.0M 988.0K 214.1M 0% /tmp/mnt
mtd:bootfs jffs2 4.4M 3.3M 1.1M 75% /bootfs
tmpfs tmpfs 215.0M 988.0K 214.1M 0% /tmp/mnt
mtd:data jffs2 8.0M 736.0K 7.3M 9% /data
tmpfs tmpfs 215.0M 988.0K 214.1M 0% /tmp
/dev/mtdblock9 jffs2 47.0M 6.5M 40.5M 14% /jffs
/dev/sda ext4 58.4G 3.0G 52.5G 5% /tmp/mnt/1234-ASUS-AC86U
tmpfs tmpfs 215.0M 988.0K 214.1M 0% /www/index_style.css
tmpfs tmpfs 215.0M 988.0K 214.1M 0% /www/require/modules/menuTree.js
Team A86U/Merlin/384.19 - my final testing results are repeatable. AC86U owners, if you are getting this CRC error, then moving your J* scripts to USB may solve the gotcha for now by using the AMTM menu option in each script. Again, to be clear, all J* scripts on this AC86U were using the JFFS BEFORE ASUS decided it was a great idea to "resize from 48M to 47M"in 384.19. The setup had been running for 1+ years with no CRC or JFFS GC errors reported b/c I check the logs to see if there are issues
before any upgrade. My gut says, my AC86U needs a
@L&LD nuclear reset and then still, I'm doubting it will fix/repair/ or bypass the bad NAND blocks. I think this router is going to end up being a WAP - somewhere safe where it's not beating up the JFFS.
Game over on 384.19 + AC86U. I am sorry I do not have better news to report.
For AC86U owners who have had zero issues with 384.19 + JFFS + AMTM J* scripts on JFFS - be happy!
My gut also says hitting the JFFS hard with J* scripts may
not be in the best long term health of the AC86U's NAND even if it is SLC and even if it should "outlive me by design." I'm going to permanently move the J* scripts to the USB SSD and see what happens there going forward with a replacement router - AC86U or not.
When I go nuclear on this AC86U, I'll post if it has any impact. Any other suggestions, feel free to share! I might even wipe it down to 384.18 just to see..
I'll also be implementing the blue/green update approach I mentioned earlier. I've burned way too many hours on this issue which I feel ASUS's little JFFS resizing may have been a root cause of or for sure a contributing factor - sure looks that way from my experiences.
Stay safe, stay alive. Peace.