This memorial, to be found by the door of the Surrey Chapel, on the corner of Botolph Street and St Crispin's in Norwich, was erected by the Protestant Alliance. It was replaced by this one which was erected on the Riverside gardens by Bishops Bridge which is about the length of a football pitch away from the supposed site of the Lollard's pit.
In fact no one is absolutely sure where the Lollard's Pit was situated; Some argue it was under the Gasometer on Gas Hill, some say it lay beneath the back bar of the Bridge Public House, others put the case for below Godfrey's Store (the paint department I believe) on Riverside Road or underneath Chalk Hill House on Rosary Road.
This newer memorial was also erected by the Protestant Alliance; an organisation I am not familiar with. A Google search on Protestant Alliance brings you to a page on http://www.orangenet.org/pa.htm which is part of Orange.net which describes itself as "the hub of Orangeism on the net"
Interestingly Thomas Bilney did not consider himself a protestant. "He was to the last perfectly orthodox on the power of the Pope, the sacrifice of the Mass, the doctrine of transubstantiation and the authority of the church." He did however preach against saint and relic veneration, disapproved of the practice of pilgrimage and did not believe in the mediation of the saints. I think he also rejected the teachings of Martin Luther. (source Wikipedia)
He seems an unlikely person for Orange Men to memorialise. If you look at the Protestant Alliance web page try clicking on the links for Orange Postcards; you will find one (and maybe more) celebrating the Ulster Volunteer Force. Their memorable deeds included the McGurk's Bar bombing in which 15 civilians were killed. Like Thomas Bilney their victims were all Roman Catholic.