Appending output to a file at the same time truncating it to N lines

If your version of sed has the --in-place switch you could use that. It still does a cp and mv but it's behind the scenes so the script would look a little less messy depending on how you feel about sed syntax. Otherwise your tail (you missed one in your edit) and append method is the way I'd approach it.

Another approach using sed would be to keep your log upside down with the newest entry at the beginning.

sed -ni '1s/^/New Entry\n/;100q;p' logfile

That's all there is to it (other than getting "New Entry" to be what you want). That will keep a rolling 100 line log file in reverse chronological order. The advantage is that you don't have to know how many lines are in the file (tail takes care of this for you, but sed won't). This assumes to some extent that your log entries consist of a single line. If you're logging a significant amount of output from a script I think you're back to tail and append.

Yet another option which might work well for detailed multi-line output would be to let the filesystem do the work for you. Each log entry would be a separate file.

#!/bin/bash
# *** UNTESTED ***
logdir="/path/to/your/log/dir"

# run the script and log its output to a file named with the date and time
script > "$logdir/$(date "+%Y%m%dT%H%M%S").log"

# maintain the logs
# keep the last 100 days of daily logfiles
# (you could keep the last 100 hours of hourly logfiles, for example,
# by using -mmin and doing some math)

# the count is intended to keep old entries from being deleted
# leaving you with nothing if there are no new entries
count=$(find "$logdir" -maxdepth 1 -type f | wc -l)
if (( count > 100 )) 
then
    find "$logdir" -maxdepth 1 -type f -mtime +100 -delete
fi

In general I would recommend you don't worry about the log size. The standard way to manage this is with logrotate. Let the administrators decide how large the want the log to be. Part of being a good *nix app I believe is sticking to the common standards, people get angry when an app does something out of the ordinary.

It is inadvisable because there is no way to delete the 1st line of a file without re-writing the entire file. Depending on the file size that is an intensive operation. Even if you did want to do that the two methods for doing so (in place with seeking or unlink, recreate) will result in tail not seeing new lines or thinking all the lines are new.