Secured OpenDAP: location of .httprc file, current working directory and $HOME environment variable

[plieger]

Hi all,

I have a question about the netcdf 4.3.2 library and accessing ESGF opendap servers which are
secured with x509 client authentication.

I usually store config settings about my certificate and ssl in the .httprc file, I have several of
them at different places. In the past I used the $HOME environment variable to designate which one
to use, but this does not seem to work with netcdf 4.3.2.

As described in http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html chapter
4.12.4 HTTP Configuration, you can either set the current working directory to this place or export
the HOME variable to the directory where this file resides. I found this very useful functionality
(now I have to change the working directory all the time).

This is an example of such an .httprc file:
HTTP.SSL.VALIDATE=0
HTTP.COOKIEJAR=/someplace/.dods_cookies
HTTP.SSL.CERTIFICATE=/someplace//certs/creds.pem
HTTP.SSL.KEY=/someplace/creds.pem
HTTP.SSL.CAPATH=/someplace/esg_trusted_certificates/

Is it possible to get this back working again?

Thanks,
Maarten Plieger

[Dennis Heimbigner]

Unfortunately, I do not have an acct with ESG,
so it is going to be difficult to debug this.
Let me start by asking you to do the following:
1. append the string '#noprefetch&show=fetch&log'
to your url
2. send me the output of, say, ncdump using the modified url
=Dennis Heimbigner
Unidata

Post by plieger
Hi all,
I have a question about the netcdf 4.3.2 library and accessing ESGF
opendap servers which are secured with x509 client authentication.
I usually store config settings about my certificate and ssl in the
.httprc file, I have several of them at different places. In the past I
used the $HOME environment variable to designate which one to use, but
this does not seem to work with netcdf 4.3.2.
As described in
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html
chapter 4.12.4 HTTP Configuration, you can either set the current
working directory to this place or export the HOME variable to the
directory where this file resides. I found this very useful
functionality (now I have to change the working directory all the time).
HTTP.SSL.VALIDATE=0
HTTP.COOKIEJAR=/someplace/.dods_cookies
HTTP.SSL.CERTIFICATE=/someplace//certs/creds.pem
HTTP.SSL.KEY=/someplace/creds.pem
HTTP.SSL.CAPATH=/someplace/esg_trusted_certificates/
Is it possible to get this back working again?
Thanks,
Maarten Plieger

[Dennis Heimbigner]

Ok, I tried it on the lastest version out of github.
I used your certificate file and your .dodsrc file.
It seems to work fine for me.
So I did this command (using 4.3.2).
The '-h' was just to limit the amount of output.
ncdump -h 'http://carbon.dkrz.de/thredds/dodsC/cordex/output/EUR-11/KNMI/ICHEC-EC-EARTH/historical/r1i1p1/KNMI-RACMO22\
E/v1/day/tas/v20140313/tas_EUR-11_ICHEC-EC-EARTH_historical_r1i1p1_KNMI-RACMO22E_v1_day_19500101-19501231.nc'

It produced the output below, as expected.
So not sure why it fails for you. My guess
is that there is some kind of firewall/proxy
at your site that is interfering.
Also, check that your libcurl supports https.
=Dennis Heimbigner

netcdf tas_EUR-11_ICHEC-EC-EARTH_historical_r1i1p1_KNMI-RACMO22E_v1_day_1950010\
1-19501231 {
time = UNLIMITED ; // (365 currently)
bnds = 2 ;
maxStrlen64 = 64 ;
rlat = 412 ;
rlon = 424 ;
char rotated_pole(maxStrlen64) ;
rotated_pole:grid_mapping_name = "rotated_latitude_longitude" ;
rotated_pole:grid_north_pole_latitude = 39.25f ;
rotated_pole:grid_north_pole_longitude = -162.f ;
.....
}
Hi Dennis,
ncdump -h
"http://carbon.dkrz.de/thredds/dodsC/cordex/output/EUR-11/KNMI/ICHEC-EC-EARTH/historical/r1i1p1/KNMI-RACMO22E/v1/day/tas/v20140313/tas_EUR-11_ICHEC-EC-EARTH_historical_r1i1p1_KNMI-RACMO22E_v1_day_19500101-19501231.nc#noprefetch&show=fetch&log"
syntax error, unexpected WORD_STRING, expecting WORD_WORD
context: Error { code = 404; message =
"cordex/output/EUR-11/KNMI/ICHEC-EC-EARTH/historical/r1i1p1/KNMI-RACMO22E/v1/day/tas/v20140313/tas_EUR-11_ICHEC-EC-EARTH_historical_r1i1p1_KNMI-RACMO22E_v1_day_19500101-19501231.nc#noprefetch&show=fetch&log"^;};
NetCDF: Malformed or inaccessible DAP DDS
I created a tarball with a valid .dodsrc, credential valid for 10 hours,
and a text file with an opendap link. This should work if you cd in the
ncdump -h
http://carbon.dkrz.de/thredds/dodsC/cordex/output/EUR-11/KNMI/ICHEC-EC-EARTH/historical/r1i1p1/KNMI-RACMO22E/v1/day/tas/v20140313/tas_EUR-11_ICHEC-EC-EARTH_historical_r1i1p1_KNMI-RACMO22E_v1_day_19500101-19501231.nc
If the creds.pem expires I can send you a new one. I will see if I can
make one with a longer validity period. You can see the expiry date with
openssl x509 -in creds.pem -noout -text
Thanks,
Maarten

Post by Dennis Heimbigner
Unfortunately, I do not have an acct with ESG,
so it is going to be difficult to debug this.
1. append the string '#noprefetch&show=fetch&log'
to your url
2. send me the output of, say, ncdump using the modified url
=Dennis Heimbigner
Unidata

Post by plieger
Hi all,
I have a question about the netcdf 4.3.2 library and accessing ESGF
opendap servers which are secured with x509 client authentication.
I usually store config settings about my certificate and ssl in the
.httprc file, I have several of them at different places. In the past I
used the $HOME environment variable to designate which one to use, but
this does not seem to work with netcdf 4.3.2.
As described in
http://www.unidata.ucar.edu/software/netcdf/docs/netcdf/DAP-Support.html
chapter 4.12.4 HTTP Configuration, you can either set the current
working directory to this place or export the HOME variable to the
directory where this file resides. I found this very useful
functionality (now I have to change the working directory all the time).
HTTP.SSL.VALIDATE=0
HTTP.COOKIEJAR=/someplace/.dods_cookies
HTTP.SSL.CERTIFICATE=/someplace//certs/creds.pem
HTTP.SSL.KEY=/someplace/creds.pem
HTTP.SSL.CAPATH=/someplace/esg_trusted_certificates/
Is it possible to get this back working again?
Thanks,
Maarten Plieger

_______________________________________________
netcdfgroup mailing list
http://www.unidata.ucar.edu/mailing_lists/