failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)

  • ATTENTION! As of November 1, 2020, you are not able to reply to threads 6 months after the thread is opened if there are more than 500 posts in the thread.
    Threads will not be locked, so posts may still be edited by their authors.
    Just start a new thread on the topic to post if you get an error message when trying to reply to a thread.

FlyByWire

Regular Contributor
Hi there,

running Merlin 378.56_2 I found this message:
Code:
dnsmasq-dhcp[19831]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
flooding the log since this morning 11:30. I attach a snapshot of the memory load from the gui. I have attached a USB harddisk and running just two jffs scripts to overclock (according to the wiki).


Any idea?

Thanks

FlyByWire
 

RMerlin

Asuswrt-Merlin dev
Odd, since the router does report there's still 82 MB of free RAM (plus 102 MB of cache that would be freed when required).

Try looking for more detailed info over ssh, with the following commands:

Code:
df -h
ls -lh /tmp
free
 

FlyByWire

Regular Contributor
Meanwhile I did a reboot (via gui) and the messages didn't recur, so I don't know if the that'll be helpfull...
 

RMerlin

Asuswrt-Merlin dev
Try doing the same thing after a few days (depending on how long it took the last time for the problem to occur) so you can compare. What you are seeing now is the state of your router after a fresh reboot. We'd have to compare both situations to determine if something is amiss.

One thing I do note already: you have a fairly large smb.log (19 MB). If you enabled logging through a custom config, you might want to either disable that, or move the log to the USB disk.
 

FlyByWire

Regular Contributor
If you enabled logging through a custom config, you might want to either disable that, or move the log to the USB disk.

Logging is not enabled. I did set up a parental control for one device. Do you think this might cause the large smb log? I'll keep an eye on the log...
 

RMerlin

Asuswrt-Merlin dev
Logging is not enabled. I did set up a parental control for one device. Do you think this might cause the large smb log? I'll keep an eye on the log...

I see that log is enabled by default, however mine is empty. Check what yours contain, something is causing it to fill up quite quickly, judging from its size.
 

FlyByWire

Regular Contributor
The error message didn't recur but smb.log is meanwhile 74 MB(!) showing this content:
Code:
Operate= 1
Operate= 2
Operate= 2
Operate= 2
Operate= 2
Operate= 2
Operate= 2
Operate= 2
Operate= 2
Operate= 2
continuing Operate=2 until eof...
 

mattiL

Regular Contributor
I had the same message for the first time this morning:

Nov 17 09:22:30 dnsmasq-dhcp[433]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)

It repeated for about two hours and then stopped.

The memory usage yesterday was in the region of 50% IIRC, and today it's just 20%, and the router has not restarted.

Uptime is:
17:23:52 up 9 days, 8:30, load average: 0.00, 0.01, 0.04

'free' shows:
[email protected]:/tmp/home/root# free

total used free shared buffers
Mem: 255720 54056 201664 0 2108
-/+ buffers: 51948 203772
Swap: 0 0 0


And 'df -h' shows:
[email protected]:/tmp/home/root# df -h

Filesystem Size Used Available Use% Mounted on
/dev/root 42.4M 42.4M 0 100% /
devtmpfs 124.8M 0 124.8M 0% /dev
tmpfs 124.9M 5.6M 119.3M 4% /tmp
/dev/mtdblock4 62.8M 1.7M 61.0M 3% /jffs

[email protected]_2
 

rlcronin

Regular Contributor
For what its worth, I am seeing the same message in my RT-AC68U logs from time to time. I am running Merlin 378.56_2 as well and am literally doing nothing with any special scripting or any such. No parental control, no overclocking, no nothing. Just the off the shelf Merlin build.
--
bc
 

scyto

Regular Contributor
did this get solved? I was troubleshooting why clients were not appearing on the client list or the DNS reservation list drop down for knew reservations but were getting logged in the DHCP list. I saw the errors the OP describes (i am running 380.57) with 108MB RAM free. I went to turn off anything that is intensive went to adaptive QOS and then clicked traffic manager page and the web page 404'd on me - i saw this in log when i logged back in (i.e error went away - this looks like some sort of buffer get exceeded?) also it corrected the client list being out of date and the client list on the drop down in the DHCP reservation feature being out of date (an:

Feb 18 17:54:48 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.151 00:1e:c0:10:86:8c
Feb 18 17:54:48 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.151 00:1e:c0:10:86:8c PCMWIFIG
Feb 18 17:54:48 dnsmasq-dhcp[2074]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 2s)
Feb 18 17:54:50 dnsmasq-dhcp[2074]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 8s)
Feb 18 17:54:55 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:54:55 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
Feb 18 17:54:55 dnsmasq-dhcp[2074]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 3s)
Feb 18 17:54:57 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.151 00:1e:c0:10:86:8c
Feb 18 17:54:57 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.151 00:1e:c0:10:86:8c PCMWIFIG
Feb 18 17:54:57 dnsmasq-dhcp[2074]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 1s)
Feb 18 17:54:58 dnsmasq-dhcp[2074]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 9s)
Feb 18 17:55:03 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:55:03 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
Feb 18 17:55:18 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:55:18 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
Feb 18 17:55:26 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:55:26 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
Feb 18 17:55:30 watchdog: restart httpd
Feb 18 17:55:30 rc_service: watchdog 454:notify_rc start_httpd
Feb 18 17:55:30 httpd: start httpd
Feb 18 17:55:31 hour monitor: daemon is starting
Feb 18 17:55:34 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:55:34 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
Feb 18 17:55:42 dnsmasq-dhcp[2074]: DHCPREQUEST(br0) 192.168.1.100 28:18:78:9a:78:4e
Feb 18 17:55:42 dnsmasq-dhcp[2074]: DHCPACK(br0) 192.168.1.100 28:18:78:9a:78:4e XboxOne
 

dakotacody

New Around Here
Not sure if you got your issue fixed or not. I was experiencing this and found a thread where Merlin said it looked to be a memory leak in lighttpd. I disabled AICloud via the web interface and checked that no lighttpd procs were still running. The issue was fixed for me after that. If you don't need AICloud running this may be the easiest fix. I didn't even need to reboot. I don't usually leave AICloud running but just log in via web interface when necessary. He may have fixed it in latest firmware too.

uptime
23:13:10 up 259 days, 8:19, load average: 0.00, 0.00, 0.0

Link to the other post about lighttpd http://www.snbforums.com/threads/memory-allocation-error-write-error.8562/
 

MartinK

New Around Here
I have the same problem; running the latest release 380.65; 2 clients connected; nothing enabled; no scripts, no upnp, no samba, no nothing; and still seeing an increase of memory usage over time; cleared log; the problem is currently occurring; rebooting resets the memory usage back to 18%; overtime i'm back at 45% and more.
 

spalife

Regular Contributor
Check with free
If it is due to caching, you can schedule a cron job to periodically clear the cache like every hour
cru a dropCaches "0 * * * * echo 3 > /proc/sys/vm/drop_caches"
 

Marko Polo

Senior Member
Still not solved. I am on 384.5. Any suggestions?

Code:
Jun 28 14:32:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:33:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:34:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:35:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:36:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:37:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
Jun 28 14:38:00 dnsmasq[28077]: failed to allocate 1028 bytes
Jun 28 14:38:51 dnsmasq-dhcp[28077]: failed to write /var/lib/misc/dnsmasq.leases: No space left on device (retry in 60s)
 

Marko Polo

Senior Member
Code:
[email protected]:/tmp/home/root# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                33.9M     33.9M         0 100% /
devtmpfs                124.8M         0    124.8M   0% /dev
tmpfs                   124.9M     77.9M     46.9M  62% /tmp
/dev/mtdblock4           62.8M      3.9M     58.8M   6% /jffs
/dev/sda1               234.7G    103.9G    118.9G  47% /tmp/mnt/data

It seems hat my root partition is filled up. Is it the reason? How to find out what is filling it?
 

Marko Polo

Senior Member
It seems that /dev/root should be an internal memory user has no access to. Isn't it? Something is wrong with router services, supposedly.
 

ColinTaylor

Part of the Furniture
root is always 100% because it's in ROM.

/var is symlinked to /tmp/var so we're interested in /tmp

tmpfs 124.9M 77.9M 46.9M 62% /tmp

It's not 100% but 77.9MB used doesn't sound right. There's probably another process that's trying to write a very large file somewhere under /tmp. Finding it is the challenge. Try looking for large files in /tmp and /tmp/var/log.

EDIT: Also /tmp/home/root
 

Marko Polo

Senior Member
There's probably another process that's trying to write a very large file somewhere under /tmp. Finding it is the challenge. try looking for large files in /tmp and /tmp/var/log.

Nothing interesting. var directory itself is only 900KB, all other files in /tmp are negligible. The only exception is /tmp/mnt but I suppose it is a mounting symlink, not a real directory.
 

ColinTaylor

Part of the Furniture
You could try running the following command. It's a bit ugly but it does the job. Look for any unusually large files.

find /tmp -exec ls -de {} \; | grep -v " /tmp/mnt/"

Unfortunately I suspect we're looking for a file that's failed to be written, not one that has. Were there any other "failed to write" messages in the log?
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top