I fixed the issue of visual artifacts with the level of detail by calculating ray distance in discrete units of voxel volumes. Before the distances were calculated from the origin to where ever the ray hit a side of a voxel. Furthermore, the mipmaps are per each voxel volume, so it makes even more sense to do it this way.  Here's a video in color coded level of detail display: