Entware Script idea, looking for feedback and suggestions.

[domic]

I read up on the

sync

command, and I was thinking about the possibility of writing a similar shell script that does the same thing for all config files stored in

/tmp/home/root/

to

/mnt/entware-usb/entware/root/

and back.

The purpose is to, for example, have all shell histories be stored to the USB drive where Entware is installed, so syncs it the same times and ways as the regular firmware does with syslogs and other important data.

For starters, I'm thinking to run a 'sync' during boot after everything has mounted and applications have started, and run a 'sync' before a system shutdown.

@RMerlin What times does the

sync

command run in the Asus/Merlin firmware other than boot and shutdown, to combat any data lost due to a unscheduled power failure etc?

 

[ColinTaylor]

I read up on the

sync

command, and I was thinking about the possibility of writing a similar shell script that does the same thing for all config files stored in

/tmp/home/root/

to

/mnt/entware-usb/entware/root/

and back.

The purpose is to, for example, have all shell histories be stored to the USB drive where Entware is installed, so syncs it the same times and ways as the regular firmware does with syslogs and other important data.

sync doesn't do that. Maybe you're thinking of something like rsync.

For starters, I'm thinking to run a 'sync' during boot after everything has mounted and applications have started, and run a 'sync' before a system shutdown.

The system automatically runs sync as part of the shutdown or reboot procedure.

What times does the

sync

command run in the Asus/Merlin firmware other than boot and shutdown, to combat any data lost due to a unscheduled power failure etc?

sync doesn't run as a system wide scheduled task. To do so would be hugely inefficient have a serious performance impact. Processes sync their data depending on their own needs and how they are configured. Filesystems have their own commit behaviour depending on the options they were mounted with. The biggest protection against data loss from power failure is to use journaled filesystems.

 

[Martinski]

To expand on this a bit, here are my 2 cents.

@RMerlin What times does the

sync

command run in the Asus/Merlin firmware other than boot and shutdown, to combat any data lost due to a unscheduled power failure etc?

As @ColinTaylor already pointed out, there is no such scheduled task of the sync command. However, the kernel memory management processes that maintain & control the "page cache" are also responsible for making sure that changes to file-backed memory pages due to processes performing write operations are in fact propagated to the physical disk drive. This synchronization between the in-memory page cache and the physical disk drive is referred to as "page writeback."

In general terms, when processes make calls for disk write operations, the disk itself is not immediately updated. Instead, the write operations only update/modify the page cache, and when this occurs the just-written-to pages are marked as "dirty" and added to the "dirty page list." This means that write operations are in fact deferred & accumulated in the page cache (to avoid too many frequent calls to slow & expensive physical disk I/O operations).

Now, to prevent "dirty pages" from remaining in memory for a long time, and to prevent loss of data in case of system crashes, power failures, etc., the kernel has a set of system parameters that control when & how often the dirty pages are written to disk. The ones that set up periodic checks for "page writebacks" are the following:

The "dirty_expire_centisecs" parameter defines how old a "dirty page" must be to be eligible for a page writeback. The value is given in 100th of a second. So when a page has been "dirty" longer than this threshold, the kernel will initiate a data flush to disk, and upon completion, the page is now marked as "clean" which means that this page becomes "inactive" and could potentially be evicted (i.e. freed) from the page cache if/when necessary so that more RAM pages can be made available to other processes.

The "dirty_writeback_centisecs" defines an interval in 100th of a second for how often the kernel flusher thread will wake up to check for dirty pages and perform a page writeback for each dirty page that is found to be old enough (based on the value of the "dirty_expire_centisecs" parameter).

So, bottom line is that the kernel already has a mechanism where every 2 seconds the flusher thread wakes up to check if there are any pages that have been marked as "dirty" for more than 30 seconds, and if so a page writeback is performed so that the recently modified contents of a file are written to the corresponding disk blocks.