Your theory reminds me of Al Capone getting taken down for tax evasion.
Gen 6:4
There were giants in the earth in those days; and also after that, when the sons of God came in unto the daughters of men, and they bare children to them, the same became mighty men which were of old, men of renown.