SUBMISSIONS
a20_ctd.zip and a20_ct1.zip submitted by Mary Johnson on 2009-11-05
containing WOCE and Exchange CTD data.

The following changes were made to the submission files:
  WOCE
    added '*' to the end of units line and quality flag line in header
  Exchange
    datestamp changed from '20091030SIO' to '20091030ODFSIOMCJ'
  files zipped and renamed

FORMATTED FILES
NetCDF files created using exctd_to_netcdf.pl (S Diggs)
Exchange and NetCDF files opened in JOA with no apparent problems

NOTE: NetCDF files do not contain TRANSM parameter

Working directory:
/data/co2clivar/atlantic/a20/a20_2003a/original/2011.11.16_CTDO_cberys

RH2_Tritium_Submission.xlsx submitted by Susan Piercy on
behalf of Bill Jenkins on 2010-08-12 available under
'as received', unprocessed by CCHDO.

30 October 2009		New release of CCHDO A20/A22-2003 Data

1.     Replace all the old CTDO data files with the new ones.
2.     Bottle Oxygen data were wrong in original bottle file submitted.  
	I would replace ALL ODF parameters, so they jive with our final 
	submission.2003 CLIVAR A20/A22 CTDO and bottle data files 
	were re-generated

2003 CLIVAR A20/A22 CTDO and bottle data files were re-generated
Oct. 30, 2009.  A review of the data showed problems in the CTD
data files produced in Feb. 2004 and in the bottle data files
produced shipboard.  The problems and their resolution are
addressed below, and fixed in this distribution.

Temperature data in 2004 were reported as ITS90, when they were
actually IPTS68 data.  Additionally, a large surface CTD Oxygen
problem was discovered and fixed.  Apparently there was a program
bug when both A20 and A22 CTDO files were generated in Feb. 2004.
Surface CTD Oxygen data were off by tens of umol/kg near-surface
for both cruises, and well over a hundred umol/kg for many casts.
Reported CTD Oxygen data did not return to bottle data ranges
until ~400db (A20) or ~30db (A22) in the Feb. 2004 data files.

A22 CTD Oxygen data, produced with current ODF CTDO processing
software, resemble data values from the ship and agree fairly
well with bottle data.  However, A20 CTD Oxygen data do not agree
well with deep bottle data or shipboard CTD data.  CTDO processing
algorithms apparently evolved during A22 in such a way that deep
CTD Oxygen data produced from shipboard A20 CTD data and corrections
files are about 1.5-2% high vs. bottle data.  A snapshot of the
final shipboard A20 CTD Oxygen data was found in other data
listings generated just after corrections were finalized at the
end of A20.  These CTD Oxygen data agree much more closely with
bottle data, and were merged in with the current pressure,
temperature, salinity and transmissometer data. (As noted, only
temperature values should differ in these other data.)

Note that there was NOT a problem with CTD Temperature or Oxygen
data reported in the bottle data files. A postgresql database
was the source for the shipboard .sea files; it was never updated
with CTD Oxygen values while the program was incorrect.  However,
plots of CTD vs bottle Oxygen data revealed an offset between
them in umol/kg units, but good agreement in ml/l units.  It
was discovered that lab temperatures at analysis time were
erroneously stored in the bottle Oxygen draw temperature slot in
early versions of the bottle database.  The correct data were
uploaded into the A20 and A22 databases, and the CTD and bottle
Oxygen data now agree in umol/kg space as well.  New versions
of the .sum, .sea and _hy1.csv files were generated in Oct. 2009
and are also included with this data release.  Note that non-ODF
data are shipboard status, and probably need to be updated from
newer data files submitted by the other PIs.

Uncorrected transmissometer data are now included in these 2009
CTD data files.  A few surface CTD T/S/O spikes were removed or
reduced in this distribution by excluding surface data before
the point where sensors had equilibrated (usually before SBE
pumps had turned on) from the pressure-series data:

   A20: stations  1, 15, 37/2, 54, 61, 75, 78, 81, 85 and 87
       61/1  CTDO off at surface, ok? by about 20db
       75/1  CTDO low at surface, ok by about 10db
       85/1  CTDO very low at surface, ok by about 14db.

   A22: stations  9, 38, 52, 54/2
	9/1  CTDS still somewhat noisy, top 10db
       38/1  CTDO a bit low at surface, ok by about 16db 
       52/1  CTDO low at surface, ok by about 22db
       54/2  CTDO low at surface, top 2db only 
	  NOTE: station 54/2 was overwritten by [deeper] 54/1 at
		some point before A22 ended.  The shallow-cast
		data were found in and recovered from another
		file, and reformatted to CCHDO formats.

The following extra CTD casts were done because of problems on
the primary deep cast (where possible, notes from Console Logs
or ODF documentation are included):

   A20: station 66/1:  "deck unit lost power, sea cable
		conductor shorted, end cast at 750m [up]"
        station 66/2:  "port side winch/wire, pick up top
		750+m of water samples from aborted cast 1"

   A22: station 51/3 (deep cast/5052db):  precedes 51/2 in time,
		"all bottle vents left open, no usable samples
		taken"  [NOTE: documentation says orig.  51/1 was
		re-named 51/3. CTD trips reported with 51/3
		match the original trip data.]  
	station 51/2: (deep cast/4020db): "repeat cast"
		[NOTE: all bottle data are reported as cast 2]

	station 57/2 (deep cast/5033db) - precedes 57/1 in time,
		"broken sea cable conductor/signal loss resulted
		in premature termination of 57/1 (renamed 57/2)
		at 4600db after tripping 3 bottles."
	station 57/1 (deep cast/5035db) - repeat cast [NOTE:
		all bottle data are reported as cast 1]

The following extra CTD casts were for CDOM experiments and have
not been processed beyond block-averaging; no bottles were tripped
on these casts, and no sample logs were filed.  Note that these
casts typically preceded the deep CTD casts in time, despite a
cast number that was later in sequence.

     A20: stations  37/2  58/2   

     A22: stations  14/2  41/2  54/2

The following CTD casts with NO bottle data have been eliminated
in this distribution, with reasons explained below:

     A20: station 89/1  The last station for A20 was 88; this was
		an on-deck test in port after A20 finished. It
		should not have been included in any distribution.

     A22: station 51/1	Renamed 51/3; CTD data in files for
		51/1 and 51/3 are identical; however, CTD trips
		stored with cast 1 do NOT match up with trips 
		logged during cast acquisition (39 vs 36 bottles,
		different depths).

ODF documentation remains the same, since no PTSO corrections
were changed.  However, the documented "extra" high-order
polynomial corrections to CTD Oxygens as a function of Pressure
were never applied to the A20/A22 data in CTD files, in 2004
or now.  Polynomial coefficients were calculated in ml/l units,
but applied to umol/kg values in the CCHDO CTD files, and
essentially had no effect on the 2004 CTD data.

These data issues were discussed with Dr. James H. Swift,
Principal Investigator of the CTD/Hydrographic Support for
A20/A22.  He agreed that updating the data was appropriate for
this data set, and suggested we include documentation that
identified the CTD data files with surface oxygen problems. 

CTD/Hydrographic Support Principal Investigator:
	Dr. James H. Swift
	jswift@ucsd.edu

SIO/STS/ODF Data Manager:
	Kristin M. Sanborn
	ksanborn@ucsd.edu

SIO/STS/ODF post-cruise Data Analysts for A20/A22 data:
	Mary Carol Johnson		Parisa Nahavandi
	mcj@ucsd.edu			pnahavandi@ucsd.edu

please find attached the
tritium spreadsheets for A20 and A22 (2003). Please let me know if there
are any questions about them.

2009.03.28  DBK
Merging Notes for a20_2003a bottle file


The following parameters were merged using
the new merge_exchange_bot.rb created by
Justin Fields.  Parameters were merged
into exchange bottle file only at this 
time, the WOCE formatted bottle file
is not up to date.

ALKALI:
These data were sent by Alex Kozyr
on 2008.07.28.  Header TALK was 
edited to ALKALI.  Data were 
merged with no problems.  


TDN, DOC:  
These data were sent by Alex Kozyr
on 2008.02.15. Data were merged with 
no problems.  


DELC14, DELC13:
Delc14 and error, and delc13 data
were sent by Bob Key on 2008.04.17
Header was edited to reflect CCHDO
parameter names and units were added
to file before merging.  Missing values
were edited from -9 to -999 before merging.
Data were merged with no apparent problems.

**NOTE:These data were remerged after it was
brought to our attention that delc13 had
incorrect precision on our parameter table.

HE, DELHE3, NEON:
DELHE3,HELIUM,NEON and their associated
error and flag values were sent by 
Bill Jenkins on 2007.11.19.  Headers
names were edited to CCHDO parameter
names and units were added.  Data
were merged with no apparent problems.

**NOTE: It should be noted that the data history
reflects an entry by Bill Jenkins for TRITUM
(tritium) data.  This entry pertains to 
the previous occupation of line A20 (1997).
There were no TRITUM data submitted for this
cruise.  There was mention of drawing a sample
for analysis in the bottle sampling protocol 
in the doc file for this cruise, however there
is no mention that samples were drawn and
analyzed for tritium.


BRDU:
BrdU data were sent by Craig Carlson on
2006.11.  File was edited for use in merge
code:  line/feed carriage returns were edited into
file, header parameters and units were edited
to reflect CCHDO parameter names, missing values
for data and flags were added to previously blank
entries. Flags for valid values were edited from
"1" to "2".  Several stations had incorrect cast
number associated with the station/cast/sample
values :  edited station 20/2 to 20/1 for samples
7, 5, 3.  Stn/cst/samp  37/-9/-9 was deleted as it 
contained no data values.  Five entries of 37/1/-9 
were deleted as they all contained no data at all.  
The duplicate entry for 37/1/6 was deleted as 
neither entry contained data.  After these edits,
data merged with no apparent problems.


DCNS:
This file contained DCNS, FUCO, RHAM, ARABI,
GLAA, GLUC, MAN data sent by Craig Carlson on
2006.11.20.  File was edited for use in merge
code:  line feed/carriage returns were added, 
all parameter and units headers were edited 
to reflect CCHDO parameters names, missing 
values and flags were added to previously blank
entries, flags for valid values were edited from
"0" to "2", however it was assumed that all values
possessing a flag "4" were truly accurrate and 
were left as-is.  Several stations had incorrect cast
number associated with the station/cast/sample
values :  edited station 20/2 to 20/1 for samples
7, 5, 3.  Stn/cst/samp  37/-9/-9 was deleted as it
contained no data values.  Five entries of 37/1/-9
were deleted as they all contained no data at all.
The duplicate entry for 37/1/6 was deleted as
neither entry contained data.  Entire file was
edited after these changes, to contain only data
necessary for the merge code to operate.  After these 
edits, data merged with no apparent problems.


BACT:
BACT, PEUK, SYNN, PROC data sent by Craig Carlson on
2006.11.20.  File was edited for use in merge
code:  line feed/carriage returns were added, 
all parameter and units headers were edited 
to reflect CCHDO parameters names, missing 
values and flags were added to previously blank
entries, flags for valid values were edited from
"1" to "2", deleted all lines which contained
no cast, station, sample numbers which also held
no values for bacteria data, deleted all lines
which contained duplicate sample numbers and
black spaces for values.  Edited cast from 2
to 1 for station 20 samples 7,5,3.  Stasions
63, 64, 65, 51 all contained cast 2 and were 
changed to cast 1 to match original bottle
and summmary files.  Entire file was
edited after these changes, to contain only data
necessary for the merge code to operate.  After these 
edits, data merged with no apparent problems.


CDOM:
CDOM data for 325, 340, 380, 412 nm and spectral
slope CDOMSL, CDOMSN were submitted by Norm Nelson
on 2005.04.20.  The header was edited to reflect
CCHDO parameter names.  Fewer samples were submitted
in the file than contained in the CCHDO bottle file.
Missing values were added as such into the bottle file.
Values for CDOMSL and CDOMSN at station 37 cast 1
samples 34, 35 were reported as NaN's with flags of "2".  
In order to merge the rest of the data, these values 
were changed to -999.0 with a flag of "5" (not reported).  
An email  was sent to  Norm Nelson requesting further 
advice on how to report these two values.  Data were 
merged with no apparent problems.

-------------
Final file was loaded into JOA without error and plots
of all parameters were made to identify any errors as
a result of merging.  None were apparent.  

Final file containing all merged parameters named
a20_2003a_mergd_doc_tdn_alk_hene_c1314_bct_cns_bdu_cdom_hy1.csv
indicates the order in which parameters were 
merged.  This file was renamed a20_2003a_hy1.csv, citation
information prepended to the top of the file, and file 
was placed back online.

NOTE:  this file did not convert to netcdf due to unknown
errors. 
At this time, the WOCE formatted bottle file is not 
up to date with these merged and added parameters.  I 
will attempt to make as complete a file as possible 
using the new exchange_to_woce conversion code created
by Justin Fields and indicate to Jerry Kappa when
this has been successfully completed.  

Here are the final TALK data from Andrew. I will send the metadata file as soon as I get it from Andrew ti include it in the report. Please, merge the TALK numbers ASAP as these data are due so much behind. 

I resubmitted the data to you today (2 files) via the web site. Format of these may be easier for you to merge and the data now has the QC flags.

Citations have been added to all online files for a20_2003a.   
File was pre-pended to the comment lines
at the beginning of the bottle exchange file and zipped together with all other product files (all ctd, and bottle
netcdf).  All files are online.

Helium Submission.csv Type:  Status: public
Name: Jenkins, William J
Institute: WHOI
Country: USA
Expo:316N200309 Line: A20
Date: 2003-09-22 
Action:Place Online
Notes:

File: A20_BACT_11-20-06.txt  Type: txt Status: Public
Name: Carlson, Craig 
Institute: University of California Santa Barbara 
Country: uSA
Expo:316N200309 Line: A20_2003
Date: 03/10
Action:Place Data Online
Notes:
Microbial Abundance data from A20

File: A20_DCNS_11-20-06.txt  Type: txt Status: Public
Name: Carlson, Craig 
Institute: University of California Santa Barbara 
Country: USA
Expo:316N200309 Line: A20-2003
Date: 10/2003
Action:Place Data Online
Notes:
Dissolved combined Neutral Sugar data for A20

Here are several additional ancillary data that accompany the core CDOM data (already submitted) for A20 and A22.     They include concentrations of microbes in the upper 250 m, bromodioxyuridine incorporation rates (proxy for microbial production) and concentrations of dissolved combined neutral sugars for A20.  I have also included brief standard operating procedures for each parameter. Again these are level III ancillary data to the bigger CDOM data set. These data analyses are extremely labor intensive to generate and were just recently completed, QC'd and finalized. I am not sure how they are to be incorporated into the larger data sets but wanted to make sure data center received these final data.

Here is the documentation for DOC collection and analyses.  Let me know if you need any other info

We have also completed the helium analyses for the A20 and A22 cruises, and I hope to submit those results shortly. I had hoped to complete the data processing prior to the Ocean Sciences meeting, but may have to do it afterward.

Here is the documentation for DOC collection and analyses.  Let me know if you need any other info.

Final CDOM Data from 2003's A20 and A22 lines.

Last week, William Smethie sent changes to the A20 (2003) CFC data for stations 38 and 40.  Today when I began to make changes to my files, I realized that I needed to do some work to make the appropriate changes to station 38...  There are several bottles where both RSMAS and Lamont took samples (i.e., duplicates).  These are usually averaged and given a flag of 6, or in the case where one lab's sample is questionable and one is good, the good one is used.  This means that rather than flagging all of station 38 CFCs with a quality byte of 3, there are several bottles where the RSMAS data can be used. So, in a nutshell, I will be re-submitting data for station 38 this week, as soon as I make the changes.  

A20 and A22 (2003):
I have the final TCO2 data from Marilyn Roberts, I merged these data into WHPO format, made our QA-QC work, and sent the data to CCHDO on June 22, I have automatic confirmation on 
receiving these data. I've checked CCHDO web site for A20/A22 and noticed that the final TCO2 data were incorporated to the bottle data files. 

Andrew Dickson is still working on his Total Alkalinity data for A20/A22 sections. I've sent Andrew a message a few days ago with question about a status of his TALK data, but did not have a reply yet. The TALK numbers in CCHDO bottle data file are preliminary and will be adjusted as soon as I get the data from Andrew and work on merging and QC.

This is information regarding line: A20 2003
ExpoCode: 316N200309
Cruise Date: 2003/09/22 - 2003/10/20
From: WILLEY, DEBRA
Email address: dwilley@rsmas.miami.edu
Institution: UNIVERSITY
Country: USA


The file:  A20_2003_CFCs_FINAL_headers.dat - 183300 bytes
has been saved as:  20041111.130455_WILLEY_A20 2003_A20_2003_CFCs_FINAL_headers.dat
in the directory:  20041111.130455_WILLEY_A20 2003

The data disposition is:
     Public  

The bottle file has the following parameters:
     CFC-11, CFC-12, CFC-113, CCL4

The file format is:
     Plain Text (ASCII) 

The archive type is:
     NONE - Individual File 

The data type(s) is:
     Bottle Data (hyd)

The file contains these water sample identifiers:
     Cast Number (CASTNO)
     Station Number (STATNO)
     Bottle Number (BTLNBR)

WILLEY, DEBRA would like the following action(s) taken on the data:
     Merge Data
     Place Data Online

Any additional notes are:
Ascii file of CFCs from A20 2003.  Generally, odd stations were sampled and analyzed by UM/RSMAS and even stations sampled and analyzed by Lamont.    

Edits made to CTD files for a20_2003a:
- Edited all station file header line parameters "No. Headers" to "NO. HEADERS"
- Edited date parameter in headers of station files 30, 33, 36, 45, 58, and 66 to match sumfile date.
- Ran wctcvt with no errors.
- Converted files to both exchange and netcdf with no errors.
NOTE- station file 89 cast 1 is an empty file.  This is true of original file obtained from data originator as well.  Inquiry on this station file will be sent to data originator. At present it will be removed from the ctd zip file and converted format files online.

I have fixed the error found by Alex Kozyr in the A20_2003a bottle file described in the email below.  The original data PI's file did not have this error in it and the entire sample/bottle line of data was replaced with the original line.  It is unclear at this point what caused the erroneous values in the online woce file.

The new file was checked with wocecvt to find no errors and was placed online.  original file was renamed and moved to the original directory.

I have submitted 2 files to WHPO with the corrected TCO2 values for merging to the master file. I hope you like the format of the files. The quality flags changed a bit too, so please merge the flags as well. However, I have a few questions to the TCO2 PI regrading some quality flags, so they might change in a future. TALK corrected values will be sent to CDIAC in a few weeks by Andrew Dickson per our conversation today. 

The file:	a20tco2_whpo.txt - 131976 bytes
has been saved as:	20040621.084208_KOZYR_A20_a20tco2_whpo.txt
in the directory:	20040621.084208_KOZYR_A20

The data disposition is:
	Public  

The bottle file has the following parameters:
	TCO2

The file format is:
	WOCE Format (ASCII) 

The archive type is:
	NONE - Individual File 

The data type(s) is:
	Bottle Data (hyd)

The file contains these water sample identifiers:
	Cast Number (CASTNO)
	Station Number (STATNO)
	Bottle Number (BTLNBR)
	Sample Number (SAMPNO)

KOZYR, ALEX would like the following action(s) taken on the data:
	Merge Data

Any additional notes are:
	This is a file with corrected parameters of TCO2 for section A20. A22 is comming soon.

I have submitted my final A20/A22 data to  WHP, and have included 
revised documentation with it.  The documentation includes discussion of 
the final data processing, including recalculation of the TCARBN files 
using bottle salinities, as well as our secondary standards (CRMs) 
corrections.  (the Documentation section) has the original cruise report.  

The CTD data for A22 (2003) are now available on-line through all links from the whpo.ucsd.edu webpage.  In addition, ted and I have fixed all of the normal links so that both the A20 and A22 cruises (both versions of each) are accessible from all of the normal links on the WHPO website.

I have placed the A20 2003 data online with a webpage. Once we've straightened out the datahist and table generation situation, we will be able to place this page in the online tables for the Atlantic.

WEBPAGE: http://whpo.ucsd.edu/data/co2clivar/atlantic/a20/a20_2003a/index.htm

WOCE Formatted Files: 
CTD files were format corrected to include the new WHPO generated expocode as was the sumfile. the bottle file had incorrect NO_DATA values and was reformatted as well. The new expocode was included in the bottle file for consistency.

WHP-Exchange files: 
The Sumfile from ODF has minor formatting issues which preclude the translation of the bottle and CTD data into WHP-Exchange. I will attend to these problems when I return on Tuesday, 2/17/04.

Format for A22 2db pressure-series downcast CTD data:

Pressure (decibars)
Temperature (ITS-90 Deg C)
Salinity (PSS-78)
Dissolved O2 (uM/kg)
Potential Temperature (ITS-90 Deg C)
Sigma Theta
Transmissometer (0-5Volts)