gnuplot: load datafile 1:1 into datablock

How can I read a datafile as-is (or 1:1) into a datablock? And how could I do this platform independently? My attempt so far:

### load datafile "as is" into datablock for different platforms

FILE = 'Test.dat'

if (GPVAL_SYSNAME[:7] eq "Windows") {          # "Windows_NT-6.1" is shown on a Win7 system
    load "< echo $Data ^<^<EOD & type ".FILE
}
if (GPVAL_SYSNAME eq "Linux") {                # that's shown on a Raspberry
    load '< echo "\$Data << EOD" & cat '.FILE
}
if (GPVAL_SYSNAME eq "Darwin") {               # this was shown on a MacOS Sierra 10.12.6
    # how to load a datafile into datablock under MacOS?
}

print $Data
### end of code

What is the value of GPVAL_SYSNAME on a Win10, other Linux, and other MacOS systems? How many if statements would I need to cover all common systems? At least under Windows the console window is flashing. How could I possibly surpress this?

My thoughts behind reading data into a dataset are the following:

if you have data on a very(!) slow server path
if you have relatively large datafiles
if you fit and plot multiple curves from several files

For example something like:

FILE1 = '\\SlowServer\blah\BigDataFile.dat'
FILE2 = '\\SlowerServer\blah\BiggerDataFile.dat'
FILE3 = '\\SlowestServer\blah\BiggestDataFile.dat'
fit f(x) FILE1 u 1:2 via a,c,d,e
fit g(x) FILE2 u 2:3 via f,g,h,i
fit h(x) FILE3 u 2:3 via j,k,l,m
plot FILE1 u 1:2:3 w l, \
     '' u (function($1)):(function($2)):3 with <whatever>, \
     FILE2 u 4:5:6 w l, \
     '' u 1:2:3 w l, \
     FILE3 u 7:8:9 w l, \
     '' u 1:2:3 w l , \
     <and more...>

My questions:

Everytime you plot or fit FILE and '', will the content of FILE be loaded again and again or will it be kept in memory?
If you zoom in, e.g. in an interactive wxt terminal, it looks to me as if the files need to be loaded again. Is this true?
If the files are loaded again and again, wouldn't it be best practice to load files once into datablocks once at the beginning and then work with these datablocks?

Any explanations, limitations, pros & cons and comments are appreciated.

Addition:

(partial answer, but with new issue): For the systems Windows,Linux and MacOS the following seems to work fine. Linux and MacOS are apparently identical.

if (GPVAL_SYSNAME[:7] eq "Windows") { load '< echo $Data ^<^<EOD & type "Test.dat"' }
if (GPVAL_SYSNAME eq "Linux" )      { load '< echo "\$Data << EOD" & cat "Test.dat"' }
if (GPVAL_SYSNAME eq "Darwin")      { load '< echo "\$Data << EOD" & cat "Test.dat"' }

However, if I want to call this construct from an external gnuplot procedure "FileToDatablock.gpp" it reproduceably crashes gnuplot under Win7/64 (haven't had a chance to test Linux or MacOS).

"FileToDatablock.gpp"

### Load datafile "as is" 1:1 into datablock for different platforms
# ARG1 = input filename
# ARG2 = output datablock
# usage example: call "FileToDatablock.gpp" "Test.dat" "$Data"

if (ARGC<1) { ARG1 = "Test.dat" }
if (ARGC<2) { ARG2 = "$Data" }
if (GPVAL_SYSNAME[:7] eq "Windows") { load '< echo '.ARG2.' ^<^<EOD & type "'.ARG1.'"' }
if (GPVAL_SYSNAME eq "Linux" ) { load '< echo "\'.ARG2.' << EOD" & cat "'.ARG1.'"' }
if (GPVAL_SYSNAME eq "Darwin") { load '< echo "\'.ARG2.' << EOD" & cat "'.ARG1.'"' }
### end of code

And the file which calls this procedure:

### load datafile 1:1 into datablock
reset session

# this works fine under Win7/64
FILE = "Test.dat"
DATA = "$Data"
load '< echo '.DATA.' ^<^<EOD & type "'.FILE.'"'
print $Data

# this crashes gnuplot under Win7/64
call "tbFileToDatablock.gpp" "Test.dat" "$Data"
print $Data
### end of code

What's wrong with this? Can anybody explain why and how to solve this issue?

It is possible to read a file into a datablock, provided you know the input data format. For example, you have a file MyFile1 with numbers in 3 columns which you want to read into datablock MyBlock1, then plot in 3 ways:

set table $MyBlock1
   plot "MyFile1" using 1:2:3 with table
unset table
plot $MyBlock1 using 1:2 with points
plot $MyBlock1 using 2:3 with points
plot $MyBlock1 using 1:3 with lines

This avoids reading the file several times, and should presumably work on any platform. Rather than doing this, I imagine it would be simpler to just copy your files from your slow filesystem to a local filesystem.

The idea is to get a datafile as is (1:1) into a datablock, including commented lines or empty lines, etc. As far as I know, there seems to be no simple and direct platform-"independent" gnuplot command for this. In some cases it might be advantageous to have data in datablocks (which are availabe since gnuplot 5.0), because you can simply address lines by index (only since gnuplot 5.2), e.g. $Data[7], or loop data forward and backwards, which you cannot do easily with data from a file.

Here is finally a solution which is acceptable for me and it seems to work on Windows and Linux (tested Windows 7 and 10 and Ubuntu 18.04.4). I couldn't test on MacOS, but I assume the command will be identical with Linux and it will work for MacOS as well. I don't know about other operating systems (feedback appreciated).

Code:

### load data file as is 1:1 into datablock
reset session

FileToDatablock(f,d) = GPVAL_SYSNAME[1:7] eq "Windows" ? \
                       sprintf('< echo   %s ^<^<EOD  & type "%s"',d,f) : \
                       sprintf('< echo "\%s   <<EOD" & cat  "%s"',d,f)     # Linux/MacOS

FILE = 'Test.dat'
load FileToDatablock(FILE,'$Data')

print $Data
### end of code

Data file: (Test.dat)

# This is a test file
1.1    1.2
2.1    2.2
3.1    3.2

# another commented line
4.1    4.2
5.1    5.2
# some empty lines will follow


  6.1    6.2   # some spaces at the beginning
7.1    7.3
# end of datafile

Result: (as expected $Data is 1:1 equal to Test.dat)

# This is a test file
1.1    1.2
2.1    2.2
3.1    3.2

# another commented line
4.1    4.2
5.1    5.2
# some empty lines will follow


  6.1    6.2   # some spaces at the beginning
7.1    7.3
# end of datafile

gnuplot: load datafile 1:1 into datablock

Related

Recent Posts