I traced down the issue but am stuck right now 

float visTerm = SmithJointGGXVisibilityTerm( NdotL, NdotV, roughness ); -> causing the Black outer circle

float normTerm = GGXTerm(NdotH, roughness); -> causing the Black Dot

NdotL and NdotH are probably causing the problem. changing them to 0.5 removes the black areas.

removing directSpecualr in the specular calculation and only using indirect Specular seems to remove the Black Areas. but still I don't think this is the right solution but maybe help fixing it.

Update:

Reducing Gloss to<1 fixes part of the Problem

There is not such a black Point in the reflection HDR CubeMap

And the Reflective area is also only a round Shape witch ends at about 70° (at Glass 0.99, lower gloss less black outer area) After that its again Black