1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

RT-AC-86U - Help with No Free Space Left for GC (Solved for now)

Discussion in 'ASUS AC Routers & Adapters' started by Chipster, Apr 19, 2019.

  1. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Hello SNB land. Long a Merling fan - but I have this 86U that's still on Asus code - version 384.32799 - and then to 384.45149.

    Last few days I'm getting this in the logs - and Google hasn't helped much. Via telnet - nothing unusual about JFFS - everything seems OK. But I'm not a good Linux nerd.

    Can anyone assist with these errors? They repeat forever..

    Apr 19 09:16:10 kernel: jffs2: jffs2_reserve_space_gc of 196 bytes for garbage_collect_dnode failed: -28
    Apr 19 09:16:10 kernel: jffs2: Error garbage collecting node at 01bd5ca0!
    Apr 19 09:16:32 kernel: jffs2: Argh. No free space left for GC. nr_erasing_blocks is 0. nr_free_blocks is 0. (erasableempty: yes, erasingempty: yes, erasependingempty: yes)
    Thank you for any help!
    Chipster
     
  2. OzarkEdge

    OzarkEdge Very Senior Member

    Joined:
    Feb 14, 2018
    Messages:
    1,240
    Location:
    USA
    Sorry, I can't help with the log entries. I would upgrade to the new release 3.0.0.4.384.45713, reset to defaults, re-configure from scratch (no restore)... and hopefully leave the issue behind.

    Maybe you are using Trend Micro features in 45149 and its crashing is messing with the logging storage.

    OE
     
  3. RMerlin

    RMerlin Super Moderator

    Joined:
    Apr 14, 2012
    Messages:
    30,248
    Location:
    Canada
    What is the size of your Traffic Analyzer database?

    Code:
    ls -l /jffs/.sys/TrafficAnalyzer/ -ah
    
    If it's larger than 30 MB, then for some reason maintenance isn't being done on it. Try running it manually:

    Code:
    TrafficAnalyzer -d 30720
    TrafficAnalyzer -e
    
    Wait 5-10 minutes, then check again the size.
     
    L&LD likes this.
  4. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Thanks for the tricks Mr. Merlin!

    But it appears to be 30M based on that command:

    Code:
    [email protected]:/tmp/home/root# ls -l /jffs/.sys/TrafficAnalyzer/ -ah
    drw-rw-rw-    2 admin    root           0 Apr 18 03:30 .
    drwxrwxrwx    4 admin    root           0 Nov 28 21:55 ..
    -rw-rw-rw-    1 admin    root       30.0M Apr 19 20:00 TrafficAnalyzer.db
    -rw-rw-rw-    1 admin    root       15.0M Apr 18 03:30 TrafficAnalyzer.db-journal
    
    And the other command:

    Code:
    [email protected]:/tmp/home/root# TrafficAnalyzer -d 30720
    table_main-1: over size 30720, timestamp=1554098400
    start to delete some rules from /jffs/.sys/TrafficAnalyzer/TrafficAnalyzer.db because of over size
    SQL error: disk I/O error
    
    While the last command gave no feedback at the command line..

    Code:
    [email protected]:/tmp/home/root# TrafficAnalyzer -e
    [email protected]:/tmp/home/root#
    
    And the system log continues to pump out the OP errors by the 100's..
     
  5. ColinTaylor

    ColinTaylor Part of the Furniture

    Joined:
    Mar 31, 2014
    Messages:
    8,309
    Location:
    UK
    What do you get from this command?

    df -h
     
  6. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Hi OE.. Just downloaded v45713. Reconfigure from scratch sounds like a good Saturday pastime - good thing I keep scrupulous notes.

    I don't have any Trend Micro stuff switched on..

    But also hoping to learn some things as I muck about with Telnet... and some of RM suggestions.

    The issue isn't completely killing the router... at least not yet.

    Thanks!
     
  7. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Oh yeah.. found that before asking here - nothing weird (just a minute ago). Could I have dead flash?:

    Code:
    [email protected]:/tmp/home/root# df -h
    Filesystem                Size      Used Available Use% Mounted on
    ubi:rootfs_ubifs         77.3M     55.0M     22.3M  71% /
    devtmpfs                215.0M         0    215.0M   0% /dev
    tmpfs                   215.1M    232.0K    214.9M   0% /var
    mtd:bootfs                4.3M      3.2M      1.0M  76% /bootfs
    mtd:data                  8.0M    580.0K      7.4M   7% /data
    tmpfs                   215.1M      2.5M    212.5M   1% /tmp
    /dev/mtdblock8           48.0M     36.5M     11.5M  76% /jffs
    [email protected]:/tmp/home/root#
    
    Edit:

    And the error logs continue:

    Code:
    Apr 19 21:21:32 kernel: jffs2: Argh. No free space left for GC. nr_erasing_blocks is 0. nr_free_blocks is 0. (erasableempty: yes, erasingempty: yes, erasependingempty: yes)
    Apr 19 21:21:32 kernel: jffs2: jffs2_reserve_space_gc of 196 bytes for garbage_collect_dnode failed: -28
    Apr 19 21:21:32 kernel: jffs2: Error garbage collecting node at 01bd5ca0!
    Apr 19 21:21:32 kernel: jffs2: Argh. No free space left for GC. nr_erasing_blocks is 0. nr_free_blocks is 0. (erasableempty: yes, erasingempty: yes, erasependingempty: yes)
    Apr 19 21:21:32 kernel: jffs2: jffs2_reserve_space_gc of 196 bytes for garbage_collect_dnode failed: -28
    Apr 19 21:21:32 kernel: jffs2: Error garbage collecting node at 01bd5ca0!
    Apr 19 21:22:32 kernel: jffs2: Argh. No free space left for GC. nr_erasing_blocks is 0. nr_free_blocks is 0. (erasableempty: yes, erasingempty: yes, erasependingempty: yes)
    Apr 19 21:22:32 kernel: jffs2: jffs2_reserve_space_gc of 196 bytes for garbage_collect_dnode failed: -28
    Apr 19 21:22:32 kernel: jffs2: Error garbage collecting node at 01bd5ca0!
    Apr 19 21:22:32 kernel: jffs2: Argh. No free space left for GC. nr_erasing_blocks is 0. nr_free_blocks is 0. (erasableempty: yes, erasingempty: yes, erasependingempty: yes)
    Apr 19 21:22:32 kernel: jffs2: jffs2_reserve_space_gc of 196 bytes for garbage_collect_dnode failed: -28
    Apr 19 21:22:32 kernel: jffs2: Error garbage collecting node at 01bd5ca0!
    

    Chipster
     
    Last edited: Apr 19, 2019
  8. RMerlin

    RMerlin Super Moderator

    Joined:
    Apr 14, 2012
    Messages:
    30,248
    Location:
    Canada
    Delete that 15 MB journal file, that shouldn't be there (and it's from yesterday anyway).
     
  9. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Well... I tried to... Wrong command? ¯\_(ツ)_/¯

    Code:
    [email protected]:/jffs/.sys/TrafficAnalyzer# rm TrafficAnalyzer.db-journal
    rm: can't remove 'TrafficAnalyzer.db-journal': No space left on device
    [email protected]:/jffs/.sys/TrafficAnalyzer#
    
    Edit: After a reboot - the file dates went back to yesterday:

    Code:
    [email protected]:/jffs/.sys/TrafficAnalyzer# ls -l
    -rw-rw-rw-    1 admin    root      31466496 Apr 18 03:30 TrafficAnalyzer.db
    -rw-rw-rw-    1 admin    root      15752624 Apr 18 03:30 TrafficAnalyzer.db-journal
    
     
    Last edited: Apr 19, 2019
  10. ColinTaylor

    ColinTaylor Part of the Furniture

    Joined:
    Mar 31, 2014
    Messages:
    8,309
    Location:
    UK
    Your jffs partition is only 48MB in size. Those two files alone take up 45MB, then there's the other files that are stored there. So you'd only need a couple of uncommitted write operations to use up all the free space.

    If as Merlin suggested the TrafficAnalyzer.db-journal is not needed I'd suggest you try and remove it as soon as possible. You might need to disable the Traffic Analyzer before it will allow you to do that. Hopefully it won't reappear when you re-enable the Traffic Analyzer.

    EDIT: The output of your df command only showed 36.5MB being used, not >45MB. So that suggests that it was trying to write that journal file but failing due to lack of free space.

    EDIT 2: It beggars belief that a modern router like the RT-AC86U with 256MB of flash memory only has 48MB available for jffs. When an old router like the RT-AC68U with 128MB of flash has 63MB! /RANT OFF
     
    Last edited: Apr 20, 2019
    L&LD likes this.
  11. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Tried deleting the -journal - but got an error.

    In the meantime - I loaded Merlin 384.10.2 - set it to reformat JFFS - and now all is working. (I didn't even reset it - 384.10 picked everything up).

    The errors no longer show in the system log.

    I may flash back to Asus code out of curiosity - now that the JFFS is cleared. Seems there should be a way to format JFFS from a terminal without flashing Eric's code and using his option to format on boot.

    Thanks for everyone's help..
     
  12. Ronald Schwerer

    Ronald Schwerer Senior Member

    Joined:
    Jan 8, 2017
    Messages:
    307
    Another way to free the space (without actually deleting the file) would be:
    cat /dev/null > /jffs/.sys/TrafficAnalyzer/TrafficAnalyzer.db
     
    Last edited: Apr 20, 2019
  13. Dave Parker

    Dave Parker Senior Member

    Joined:
    Apr 22, 2015
    Messages:
    221
    Location:
    Marmaduke Ar.
    It's my understanding that when flashing from stock Asus firmware to Merlins firmware a factory default reset is not optional but necessary. Someone correct me if I'm wrong.
     
    L&LD likes this.
  14. ColinTaylor

    ColinTaylor Part of the Furniture

    Joined:
    Mar 31, 2014
    Messages:
    8,309
    Location:
    UK
    No that's not always the case.
    But on the other hand:
    In the case of the OP the firmware versions of stock and Merlin's were probably very close.
     
  15. L&LD

    L&LD Part of the Furniture

    Joined:
    Dec 9, 2013
    Messages:
    9,105
    Be very careful about formatting anything within the router via cli commands. A mistake may prove very costly. ;)

    Unless you need AiMesh support and also can't wait until RMerlin has access to the updated GPL that fixes the network devices pages, there is very little reason to stay with stock Asus firmware. In the long run, much happier user experience with fewer bugs popping up, more features (as you've seen already) and more performance too.

    All that is true, of course, if you do a full reset to factory defaults first. ;)

    Like the readme states in the RMerlin files, you downloaded. :)

    https://www.snbforums.com/threads/faq-nvram-and-factory-default-reset.22822/

    The above link is a great read on why you should be doing one. If you want the experience to be as positive as possible.

    Have a look below at my signature for further details on how to get your router fully reset using the M&M Config guide.
     
    Dave Parker likes this.
  16. RMerlin

    RMerlin Super Moderator

    Joined:
    Apr 14, 2012
    Messages:
    30,248
    Location:
    Canada
    Going from Asus to Merlin is fine. Going from Merlin to Asus will leave you with potentially corrupted DHCP reservations (I added a third field for the hostname), and VPN settings (different OpenVPN versions and settings supported). You have to either be ready to handle these issues, or else do a factory default reset.
     
    Dave Parker and L&LD like this.
  17. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    I'll be doing a factory reset soon - and type it all back in. No need for AIMesh.. so I'll keep with Merlin.
    Sooo many DHCP reservations, WOL, port forwards.. .. :(
     
  18. L&LD

    L&LD Part of the Furniture

    Joined:
    Dec 9, 2013
    Messages:
    9,105
    Screenshots, text files, spreadsheets! :)
     
  19. RMerlin

    RMerlin Super Moderator

    Joined:
    Apr 14, 2012
    Messages:
    30,248
    Location:
    Canada
    Code:
    nvram get dhcp_staticlist
    
    Code:
    nvram set dhcp_staticlist="blah"
    nvram commit
    
    Edit afterward through the webui to add hostnames if you want to.
     
    L&LD likes this.
  20. Chipster

    Chipster Occasional Visitor

    Joined:
    Aug 1, 2014
    Messages:
    10
    Well... the router was locked up this morning. I have it to reboot at 330AM every day - but it never completed.

    So.. it did the factory reset, keyed in most things - and did a reboot (from the interface). Booted good. Then I finished some more settings - and reboot again.. and it hung on "Completed"... had to power cycle.

    So now I'm confused why it won't reliably boot from a "soft" reboot..