ListView cells being cached?

Hi all,
I have a ListView with about 500 cells. Here's the XAML I'm using to define it:

<ListView
            Grid.Row="1"
            IsGroupingEnabled="True"
            GroupDisplayBinding="{Binding Header}"
            HasUnevenRows="False"
            ItemsSource="{Binding GroupedItemsSource}">
      <ListView.GroupHeaderTemplate>
        <DataTemplate>
          <ViewCell Height="75">
            <StackLayout VerticalOptions="EndAndExpand" BackgroundColor="White">
              <Frame VerticalOptions="FillAndExpand" HasShadow="false" Padding="8, 8, 4, 0">
                <controls:MyLabel x:Name="label" VerticalOptions="End" Text="{Binding Header}" TextColor="Grey" />
              </Frame>
              <BoxView HorizontalOptions="FillAndExpand" HeightRequest="1" BackgroundColor="Grey">
              </BoxView>
            </StackLayout>
          </ViewCell>
        </DataTemplate>
      </ListView.GroupHeaderTemplate>
      <ListView.ItemTemplate>
        <DataTemplate>
          <ViewCell Height="75">
            <StackLayout Padding="8,4,8,4" Spacing="0" Orientation="Vertical">
              <controls:MyLabel Text="{Binding Name}" />
              <controls:MyLabel Text="{Binding CustomValue,Converter={StaticResource MyConverter}}" />
            </StackLayout>
          </ViewCell>
        </DataTemplate>
      </ListView.ItemTemplate>
    </ListView>

The performance of my list is very bad and I was looking for reasons for this, so I tried to use the profiler.
Here I noticed than my memory usage keeps going up while I'm scrolling the listview.
It seems like for some reason each cell is cached instead of reused?

I verified this by setting a breakpoint in OnElementChanged of my custom label and this is indeed only called once for each label.

Is there a reason all cells are cached and is there a way to turn this behavior off?
I have a feeling that it only makes performance of my ListView worse...

thanks,
Thomas