I just have this in my .vimrc which lets you toggle it whatever way you want in edit mode and it stays however you leave it when you toggle modes

    " use Ctrl+L to toggle the line number counting method
    function! g:ToggleNuMode()
    	    if(&rnu == 1)
        		set nu
	    else
		    set rnu
	    endif
    endfunc
    nnoremap <C-L> :call g:ToggleNuMode()<cr>