getValuesHdfDatatype() does not support H5T_NATIVE_LONG datasets?

FESAPI
1

Hi Philippe,

(I’m using 0.16.0.0).

I have HDF5 datasets of type H5T_NATIVE_LONG (that were created by Fesapi). But I don’t see any support for this type in the function below. It used to be there but disappeared (that was before you moved to GitHub so I could not find when and why). Is there another way to discover the type of those datasets?

Thanks,
Philippe

AbstractValuesProperty::hdfDatatypeEnum AbstractValuesProperty::getValuesHdfDatatype() const
{
…
if (H5Tequal(dt, H5T_NATIVE_DOUBLE) > 0)
return AbstractValuesProperty::DOUBLE;
else if (H5Tequal(dt, H5T_NATIVE_FLOAT) > 0)
return AbstractValuesProperty::FLOAT;
else if (H5Tequal(dt, H5T_NATIVE_LLONG) > 0)
return AbstractValuesProperty::LONG;
else if (H5Tequal(dt, H5T_NATIVE_ULLONG) > 0)
return AbstractValuesProperty::ULONG;
else if (H5Tequal(dt, H5T_NATIVE_INT) > 0)
return AbstractValuesProperty::INT;
else if (H5Tequal(dt, H5T_NATIVE_UINT) > 0)
return AbstractValuesProperty::UINT;
else if (H5Tequal(dt, H5T_NATIVE_SHORT) > 0)
return AbstractValuesProperty::SHORT;
else if (H5Tequal(dt, H5T_NATIVE_USHORT) > 0)
return AbstractValuesProperty::USHORT;
else if (H5Tequal(dt, H5T_NATIVE_CHAR) > 0)
return AbstractValuesProperty::CHAR;
else if (H5Tequal(dt, H5T_NATIVE_UCHAR) > 0)
return AbstractValuesProperty::UCHAR;

return AbstractValuesProperty::UNKNOWN; // unknwown datatype...

}

2

Hi Philippe,

The reason of this change is the different variants of size of long on different platforms.
Fesapi AbstractValuesProperty::hdfDatatypeEnum assume that INT are 32 bits and LONG 64 bits.

If your H5T_NATIVE_LONG are actually integer 64 bits (gcc 64 bits for example) then I think that

if (H5Tequal(dt, H5T_NATIVE_LLONG) > 0)

should be true (long long are “most of time” 64 bits integer)

If your H5T_NATIVE_LONG are actually integer 32 bits (visual studio 64) then I think that

if (H5Tequal(dt, H5T_NATIVE_INT) > 0)

should be true (int are “most of time” 32 bits integer)

I made a test reading a 64-bit integer HDF datasets stored by Fesapi in an HDF5 file. It looks to work just fine on Windows (VS 2015 64). “dt” variable is actually indeed a H5T_NATIVE_LLONG in the “if” statements.
It may be different on Linux (gcc). The “dt” variable could actually be a H5T_NATIVE_LONG since the HDF5 documentation states “H5T_GET_NATIVE_TYPE selects the first matching native datatype from the following list:” where H5T_NATIVE_LONG comes before H5T_NATIVE_LLONG.
Still on gcc 64bit,I would have guessed H5T_NATIVE_LONG equals H5T_NATIVE_LLONG and then that there would be no problem.

Do you face an issue? Which platform are you testing?

If there is an issue, I will probably no more rely on H5T_GET_NATIVE_TYPE but on H5T_GET_CLASS + H5T_GET_SIZE + H5T_GET_SIGN

3

Hi Philippe,

I see.

Up until v 0.16.0.0 I was consistently using DiscreteProperty’s pushBackLongHdf5Array1dOfValues() and getLongValuesOfPatch() and getting AbstractValuesProperty::LONG returned for such datasets, on Win 64.

With v 0.16.0.0 I get AbstractValuesProperty::INT returned. I understand the rationale for it, but that makes things a bit inconsistent.

I could resolve that by using pushBackIntHdf5Array1dOfValues/getIntValuesOfPatch which I understand are exactly the same as pushBackLongHdf5Array1dOfValues/getLongValuesOfPatch on my platform. A significant undertaking.

That could also be resolved by replacing getValuesHdfDatatype() by something like:

bool valuesHdfDatatypeMatches (AbstractValuesProperty::hdfDatatypeEnum)

That would return ‘true’ for those 32 bit integer datasets when called passing either LONG or INT on my platform.

Exactly like H5Tequal returns positively when H5T_NATIVE_LONG or H5T_NATIVE_INT are passed, for such datasets, on my platform.

So, to summarize, no real issue there, other than loss of consistency on platforms where ‘int’ and ‘long’ are the same thing.

Thanks for your answer that helped a lot sorting this out.

Philippe

4

Hi Philippe,

I agree with the loss of consistency which you report.
I tried to solve it in this PR : https://github.com/F2I-Consulting/fesapi/pull/190

In 1.0.0.0, long (in fesapi method name) will always refer to only 64 bits integer and int (in fesapi method name) will always refer to only 32 bits integer.
I think this PR should ensure that at least on main platforms thanks to usage of LONG64/ULONG64 gsoap datatype on fesapi methods which mention “long”.

5

Hi Philippe,

Indeed that is going to remove any ambiguity.

So it looks like now I have no other choice than to move to the ‘int’ version of the API where I was using the ‘long’ one, but that’s ok.

Thanks again,

Philippe