1904 was assumed for a while as the foundation date for North Elmham Cricket Club, however 1904 was the date that the Norfolk Cricket Club presented the Mid-Norfolk Villages Cricket Shield to the Mid-Norfolk League.
The League started in 1898 and I understand that it started at the King's Head in North Elmham which suggests that NECC was key to the creation and therefore existed in 1898. They may have played their games at County School.
It is unlikely that it existed much beofre then unless it was for the landed gentry (unlikely). What happened was the Liberal government introduced 1/2 day working on Saturday and this created what we know as "The Weekend". The landowners were worried that their labourers with this new spare time on their hands would get up to mischief. So they created things to do that would keep them occupied - hence the cricket League.
So I would guess that North Elmham Cricket Club was started either in 1897 or 1898. More likely 1898.