csv-plot.xl (Conversion of CSV Plot Data to Matrix)
Author: Hirohisa Mori / joshua@globalbase.org +
[Transrate]Reiko Inoue Bendtsen c/o Suzaku Translations /
(http://www.suzaku-translations.com/) +
PROTOTYPE
xl csv-plot.xl [CSV-FILE] [CORRESPOND] [MATRIX] [X-DOTS] [Y-DOTS]
ARGUMENTS
[CSV-FILE] CSV file of plot data
[CORRESPOND] Corresponding conversion file
[MATRIX] Save destination matrix file
[X-DOTS] Number of dots in the x direction
[Y-DOTS] Number of dots in the y direction
Option
ENVIRONMENT
Agent exl [UNDEF REF (exl)]
EXPLANATION
If you have installed LANDSCAPE, you can find sample data in /usr/local/xl-gbs/xlsamples/gbs/plot1. Please refer to it when you read the following.
This script converts plot data listed in a CSV file
[CSV-FILE] to a matrix file with the format given by
[MATRIX] . The plot data converted to matrix data covers the entire data set; if some of the plots cannot be displayed on the screen, the script behaves as follows.
-
Shrink icons
-
Change icons to simple colored points if they cannot be displayed even if they are shrunk
-
Merge points and display them as an area if there are many plots within one pixel on the screen
With these operations, an almost inifinite number of plots can be displayed.
While matrix data is basically an integer-type representation, plots tend to be sequential ( real-valued) coordinate systems, for example plots against latitude/longitude. For this reason, a resolution must be specified for the matrix, and plot coordinates are converted to integers accordingly. That is, a real number coordinate system is divided into very small grids and plot positions are expressed as the grid's integer coordinate.
[X-DOTS][Y-DOTS] indicate the size of the integer grid.
Attributes assigned to one plot inside a matrix are identified by numbers starting from 0,e.g., the Nth attribute of a plot at coordinate XX,YY. Each attribute has a data type. This data type specifies how to merge plots if it becomes necessary to merge plots during the process of lowering the resolution of a matrix image ( grid). To facilitate this, the following data types have been defined. First, teh basic data types are explained.
-
mxPgTYPE_NONE
NONE type, without values
-
mxPgTYPE_STRING
Character string type
-
mxPgTYPE_INT
Real number type
-
mxPgTYPE_RGBA
8-bitfull color type with transparency channel
In addition, each basic data type has a number of more detailed sub-types.
-
mxPgNONE
NONE type
-
mxPgSTRING
Character string type. It ignores character strings of upper level nodes of a matrix and assigns new character strings.
-
mxPgSTRING_INHERIT
Character string type. It inherits character strings of upper level nodes of a matrix ( users are not permitted to use this type).
-
mxPgSTRING_ERASE
Character string type. It ignores character strings of upper level nodes of a matrix and assigns empty character strings ( users are not permitted to use this type).
-
mxPgINT_ADD
Integer type. It assigns the total value of plots to be merged in lower level nodes to upper level nodes.
-
mxPgINT_AVG
Integer type. It assigns the average value of plots to be merged in lower level nodes to upper level nodes.
-
mxPgINT_MAX
Integer type. It assigns the maximum value of plots to be merged in lower level nodes to upper level nodes.
-
mxPgINT_MIN
Integer type. It assigns the minimum value of plots to be merged in lower level nodes to upper level nodes.
-
mxPgRGBA
RGBA type. It assigns the average value of each channel color to upper level nodes.
A CSV file
[CSV-FILE] contains a simple list of attributes and does not indicate coordinates. Similarly, attributes of a CSV file has no type. For this reason, it is necessary to specify:
-
Which lines of a CSV file should be regarded as coordinate values
-
To which attributes of a matrix file the remaining lines of a CSV file should be assigned
-
Which type should be used at assignment
These specifications are made by a file specified by
[CORRESPOND] . [CORRESPOND] is an XL script file, but only few instructions actually use it. /usr/local/xl-gbs/xlsamples/plot1/gaz.xl is one of them. When you open it, you can see that the only instructions used are ?xl, Define, and gmxPgPlotField.
Variables defined by Define
<Define> ^reso 3600 </Define>
The script above defines resolution in the reso variable, in this case 3600dots/degree. In gaz.csv in the example, the coordinates are specified in lines A and B as degrees while the matrix file defines 1second as 1dot. The command below, which is placed at the end of the file, indicates the number of lines at the beginning of the csv file to be ignored,
<Define> ^StartLine 1 </Define>
In this example, the first one line is ignored.
gmxPgPlotField
The following gmxPgPlotField specifies correspondence among each line of the csv file, coordinates of the matrix file and attribute numbers. gmxPgPlotField is a function of Agentgbmx. Please refer to
XL Function or Environment (gmxPgPlotField)for the detailed explanation of the function. It is used in the following 2ways in csv-plot.xlscript.
(A)
<gmxPgPlotField id="^mtx" csv="A" merge-type="^mxPgTYPE_INT" dim="0">
^Fields ^mxPgFDIT_DD ^reso (* 180 reso) 0</gmxPgPlotField>
(B)
(B-1)
<gmxPgPlotField id="^mtx" csv="C" merge-type="^mxPgTYPE_STRING" label="5">
^Fields utf-8 ([quote type="direct"] '([field name="LABEL"] ($ ___convert)))</gmxPgPlotField>
(B-2)
<gmxPgPlotField id="^mtx" merge-type="^mxPgTYPE_RGBA" label="0">
^Fields 0 0xff000000</gmxPgPlotField>
(B-3)
<gmxPgPlotField id="^mtx" csv="D" merge-type="^mxPgTYPE_STRING" label="1">
^Fields utf-8 () test-http </gmxPgPlotField>
(A) shows a method of specifying coordinates while (B) shows a method of specifying correspondence between attributes. The following XML attributes can be used.
-
id=" ^ mtx"
This XML attribute specifies the ID of a matrix and is fixed to this character string in case of csv-plot.xl.
-
csv
This XML attribute specifies a number of lines corresponding to attributes of a csv file, A,B,C,.... This XML attribute can be omitted.
-
label
It is used in pattern (B) ; the label XML attribute specifies the attribute number of a matrix. Lines in csv XML attribute correspond to a plot attribute number specified by the label XML attribute. If the csv XML attribute is omitted, the data value given by the raw data of gmxPgPlotField is saved in the label attribute numbers of all plots.
Although the label XML attribute can be omitted, either the label or the dim XML attribute must exist.
-
dim
This XML attribute is used in pattern (A) and specifies the coordinate axis number of a matrix. As an intrinsic function of a matrix, it is possible to specify values up to the number of matrix dimensions - 1,but since plot-csv.xl supports only 2-dimensionalmatrices, the only permitted values are 0or 0specifies the x axis while 1specifies the y axis.
Although the dim XML attribute can be omitted, either the dim or label XML attribute must exist.
-
merge-type This specifies the detailed attribute type of a matrix.
Lastly, how the element data of gmxPgPlotField is specified varies depending on the data type given by the merge-type XML attribute.
-
mxPgNONE
None
-
mxPgSTRING
[Pointer to work area] [saved character code name] [conversion formula] [initialization character string: arbitrary]
If [conversion formula] is set to (), no special processing is performed on the CSV data, which is simply set to the attribute value. If [conversion formula] is given as an XL conversion formula, the CSV data converted by this formula is set to the attribute value. The example (B-1) above corresponds to this. The CSV data is stored in the "___convert" area. The actual conversion is executed when a CSV table is created for the first time; a $ symbol cannot be interpreted at this point and it is thus necessary to use direct quotes.
If no csv attributes are specified, [initialization character string: arbitrary] becomes necessary; this character string is passed to [conversion formula] and set for all plot attributes as a constant value.
-
mxPgINT_ADD mxPgINT_MAX mxPgINT_MIN
[Pointer to work area] [data type in CSV] [conversion resolution] [conversion offset] [count] [initialization character string: arbitrary]
Data in a CSV file is first interpreted as the data type indicated by [data type in CSV] , which is read as double-precision floating point data. The detailed explanation on [data type in CSV] will be given later. The data read is converted by the formula ( [read data]x[conversion resolution]+[conversion offset]). Any fractions after the conversion are omitted, so that the converted output is an integer, and the result is set as the attribute data or coodinate value of the matrix. [Conversion resolution] and [conversion offset] must be given as double-precision floating point values.
[Count] is ignored in these three data types; it is sufficient to enter 0for the attribute.
If no csv attributes are specified, [initialization character string: arbitrary] becomes necessary; this character string is given to [conversion formula] and set for all plot attributes as a constant value.
-
mxPgINT_AVG
[Pointer to work area] [data type in CSV] [conversion resolution] [conversion offset] [count] [initialization character string: arbitrary]
The method of calculating the attribute data is the same as above. mxPgINT_ADD mxPgINT_MAX mxPgINT_MIN This data and [count] is saved as attribute data, as a pair of two integers. In case of this type, in order to merge plots with higher layers of a matrix, an average of the plots is taken and set as a new plot attribute value. This average is calculated weighted by [count] and the [count] value in the higher layers is set to the total value of [count] of lower layers.
If no csv attributes are specified, [initialization character string: arbitrary] becomes necessary; this character string is passed to [conversion formula] and set for all plot attributes as a constant value.
-
mxPgRGBA
[Pointer to work area] [count] [initialization RGBA: arbitrary]
This function acquires RGBA data from CSV and assigns it to plot attributes. If it is given as hexadecimal, the format used in C language to specify pixel values, e.g. -16777216,must be used. [Count] works in the same way as integer type [count] and is used for weighting when calculating colors of higher layers. No weighting is performed if 0is specified.
If no csv attributes are specified, [initialization RGBA: arbitrary] becomes necessary; this value is passed to [conversion formula] and set for all plot attributes as a constant value.
Integer type [Data type in CSV] The following values can be specified for this type.
-
mxPgFDIT_INT
The data type of CSV is integer ( 64-bitinteger).
-
mxPgFDIT_FLOAT
The data type of CSV is double-precision floating point.
-
mxPgFDIT_DD
The CSV data type is decimal point expression of latitude/longitude, for example means 135degress 30minutes.
-
mxPgFDIT_DDMMSS
The CSV data type is sexagesimal expression of latitude/longitude, for example means 135degrees, 50minutes, 12seconds.
How to use /usr/local/xl-gbs/xlsamples/gbs/plot1
When you want to disclose plot data, it is possible to disclose it using the following procedure.
-
Copy /usr/local/xl-gbs/xlsamples/gbs/plot1 to a disclosure point. For example, if the directory is myplot,
-
edit the img.crd bibliographic data in the myplot directory.
-
Preprare the CSV file of plot data to be disclosed and save it as a file in the myplot directory, for example as myplot.csv.
-
Edit gaz.xl file for myplot.csv to match attributes.
-
Execute xl csv-plot.xl - - / myplot.csv gaz.xl img.mtx [X-DOTS] [Y-DOTS].
-
Execute xl makefile.xl - -.
[UP]
Go To Page Top
RETURN VALUE
ERRORS
REFERENCE
/usr/local/xl-gbs/xlsamples/gbs/plot1
BUGS
[UP]
Go To Page Top