'34 (and '33) engines are basically the same for rebuilding as the later engines, except there is no full length water jacket so as you discovered the block is a little more narrow than the later engines and uses a different head gasket. But as far as internals, the bearings, pistons, etc. are the same as other 201 engines ('33 has different rods and crank as it is only 190cu in).
Water pump is different as the inlet to the block is rectangular and there is no water distribution tube behind it. Internals of the pump are the same as later so rebuilding it no big deal. If you want to put a later pump on then you will need to make a custom backing plate to match the block (same mounting stud hole pattern so they will bolt up).