Fair question.
Dependence on a mala, belief that a certain number of recitations is necessary and helped by a mala, is considered in Guru Granth to be mired in duality. I do not think it likely that Guru Nanak used a mala (see the shabad below). Can I prove it? No I cannot.
We will never know for sure if Guru Nank used malas or not because we have no eye-witness accounts and no video cams for that era in history. Artists depictions are born in the imagination of the artist. And artists are influenced by the social, cultural, religious, and political cross-currents of their times, as well as their own religious upbringing. If we could know who the artists were and something of their lives we could perhaps resolve your confusion.
From this blog at
http://www.gurbani.org/articles/webart188.htm Guruji says that the mala/rosary is not essential. It is based in "stubborn mindedness" and can also lead to a false sense of piety.
What sort of rosary are we urged by the Gurbani to carry? Sublime rosary! The beads of such rosary are not made of Rudraksh, Tulsi, Chandan, etc. According to the Gurbani, this rosary and its beads is none other than one's undivided meditation on the Divine in the Heart Within; one's good deeds; one's tongue upon which the Divine Name is strung; one's subjugation of the ten senses; one's good conduct; one's noble virtues such as kindness and forgiveness; and so on. This is the rosary of the True Being (Gurmukh or a Spiritual person). Meditation and spiritual Wisdom is not possible without such wholesome inwardly concentration. Therefore, to awaken the Divine Grace within, we are time and again reminded by the Gurbani to enshrine such a sublime rosary in the Heart, and to discard the outer rosary. In other words, Gurmukhtaa (Spiritual Life) cannot be tasted unless we are spontaneously and infinitely in love with the Supreme Self Within
On Ang 503 of Sri Guru Granth Sabhi ji, Guru Nanak himself says,
ੴ ਸਤਿਗੁਰ ਪ੍ਰਸਾਦਿ ॥
Ik▫oaʼnkār saṯgur parsāḏ.
One Universal Creator God. By The Grace Of The True Guru:
ਏਕ ਨਗਰੀ ਪੰਚ ਚੋਰ ਬਸੀਅਲੇ ਬਰਜਤ ਚੋਰੀ ਧਾਵੈ ॥
Ėk nagrī pancẖ cẖor basī▫ale barjaṯ cẖorī ḏẖāvai.
In the one village of the body, live the five thieves; they have been warned, but they still go out stealing.
ਤ੍ਰਿਹਦਸ ਮਾਲ ਰਖੈ ਜੋ ਨਾਨਕ ਮੋਖ ਮੁਕਤਿ ਸੋ ਪਾਵੈ ॥੧॥
Ŧarihḏas māl rakẖai jo Nānak mokẖ mukaṯ so pāvai. ||1||
One who keeps his assets safe from the three modes and the ten passions, O Nanak, attains liberation and emancipation. ||1||
ਚੇਤਹੁ ਬਾਸੁਦੇਉ ਬਨਵਾਲੀ ॥
Cẖeṯahu bāsuḏe▫o banvālī.
Center your mind on the all-pervading Lord, the Wearer of garlands of the jungles.
ਰਾਮੁ ਰਿਦੈ ਜਪਮਾਲੀ ॥੧॥ ਰਹਾਉ ॥
Rām riḏai japmālī. ||1|| rahā▫o.
Let your rosary be the chanting of the Lord's Name in your heart. ||1||Pause||
ਉਰਧ ਮੂਲ ਜਿਸੁ ਸਾਖ ਤਲਾਹਾ ਚਾਰਿ ਬੇਦ ਜਿਤੁ ਲਾਗੇ ॥
Uraḏẖ mūl jis sākẖ ṯalāhā cẖār beḏ jiṯ lāge.
Its roots extend upwards, and its branches reach down; the four Vedas are attached to it.
ਸਹਜ ਭਾਇ ਜਾਇ ਤੇ ਨਾਨਕ ਪਾਰਬ੍ਰਹਮ ਲਿਵ ਜਾਗੇ ॥੨
Sahj bẖā▫e jā▫e ṯe Nānak pārbarahm liv jāge. ||2||
He alone reaches this tree with ease, O Nanak, who remains wakeful in the Love of the Supreme Lord God. ||2||
ਪਾਰਜਾਤੁ ਘਰਿ ਆਗਨਿ ਮੇਰੈ ਪੁਹਪ ਪਤ੍ਰ ਤਤੁ ਡਾਲਾ ॥
Pārjāṯ gẖar āgan merai puhap paṯar ṯaṯ dālā.
The Elysian Tree is the courtyard of my house; in it are the flowers, leaves and stems of reality.
ਸਰਬ ਜੋਤਿ ਨਿਰੰਜਨ ਸੰਭੂ ਛੋਡਹੁ ਬਹੁਤੁ ਜੰਜਾਲਾ ॥੩॥
Sarab joṯ niranjan sambẖū cẖẖodahu bahuṯ janjālā. ||3||
Meditate on the self-existent, immaculate Lord, whose Light is pervading everywhere; renounce all your worldly entanglements. ||3||
ਸੁਣਿ ਸਿਖਵੰਤੇ ਨਾਨਕੁ ਬਿਨਵੈ ਛੋਡਹੁ ਮਾਇਆ ਜਾਲਾ ॥
Suṇ sikẖvanṯe Nānak binvai cẖẖodahu mā▫i▫ā jālā.
Listen, O seekers of Truth - Nanak begs you to renounce the traps of Maya.
Your question about mala however is so interesting it deserves its own thread. There are many fascinating and inspiring shabads on this subject. Please do also read more at the link I posted. The author is very deeply grounded in his subject matter.