Special characters (norwegian wovels) not showing correctly in Page Layouts RRS feed

  • Question

  • We have a publishing site which surfaces content from several catalogs in an authoring site in a cross site collection publishing environment.

    We have a problem with labels for Catalog-Item Reuse Snippets containing norwegian vowels/special characters not showing properly. An example of a snippet is shown below. This contains the character ø (oslash).

    <div class="rightColPaddingUnder"><h2>Leverandørkontakt</h2>
    	<!--CS: [NavnhovedkontaktleOWSTEXT] Start Catalog-Item Reuse Snippet-->
    	<!--SPM:<cc1:CatalogItemReuseWebPart runat="server" UseServerSideRenderFormat="True" ResultType="" NumberOfItems="1" UseSharedDataProvider="True" OverwriteResultPath="False" ResultsPerPage="1"  SelectedPropertiesJson="[&#34;NavnhovedkontaktleOWSTEXT&#34;]" Title="$Resources:Microsoft.Office.Server.Search,CBSItem_Title;" Description="$Resources:Microsoft.Office.Server.Search,CBSItem_Description;" MissingAssembly="Cannot import this Web Part." ID="g_1d79f6d5_037f_43b3_ac4b_4e7bc10ff1c3" __WebPartId="{1d79f6d5-037f-43b3-ac4b-4e7bc10ff1c3}">-->
    	<!--DC:Viser verdi fra søk uten ekstra formatering.-->
    	<!--CE:End Catalog-Item Reuse Snippet-->

    The frontend messes the character up and displays this as strange A

    We have tried several things in order to fix this but are running out of leads:

    • tried to change encoding in the page layout html file from utf-8 to iso-8859-1
    • tried to replace ø with the hex value (&#248;) and the old school & + oslash + ; way of encoding characters
    • Regional settings on the server is verified to be norwegian

    The strange part is that actual content in the content type columns that we display around the page actually shows the norwegian characters correctly

    Wednesday, February 26, 2014 10:21 AM


  • Have you double checked the IIS site's .NET Globalization settings.

    File should be set to UTF-8.IIS Settings .NET Globalization

    Perhaps something went wrong in the provisioning when creating the new web application?

    If you check the SharePoint Web Services it's set to Windows-1252 which confirms your previous stack overflow reference I guess.

    Friday, February 28, 2014 8:48 AM

All replies

  • Hi ,

    Please try replacing the character  ø  with Numerical Code &#248; in your page, see if it could be displayed as expected.



    Daniel Yang
    TechNet Community Support

    • Edited by star.wars Thursday, February 27, 2014 1:07 PM
    Thursday, February 27, 2014 1:06 PM
  • Hi Daniel and thanks for the reply!

    As I said we have already tried with the Hex-value (I clarified this in the original post now). This gives the same result. I see claims elsewhere that Sharepoint Web Services delivers ISO-8859-1 encoded XML. ref. http://stackoverflow.com/questions/841491/error-when-trying-to-load-xml-from-a-sharepoint-web-services-call-into-an-asp-ne

    I don't know if this applies to what we are experiencing.

    Thursday, February 27, 2014 1:26 PM
  • Hi mellinge and thanks for the reply.

    The page layout already contains the meta tag specifying the UTF-8 encoding. This is how a typical page layout starts:

    <?xml version="1.0" encoding="UTF-8"?><!--SPG:
    Denne HTML-filen er knyttet til et SharePoint hoveddokument (ASPX-fil) med samme navn. Når filene er tilknyttet, kan du ikke redigere ASPX-filen, og å gi nytt navn til, flytting eller sletting, blir ikke gjort gjeldende.
    Hvis du vil bygge hoveddokumentet direkte fra denne HTML-filen, fyller du ut innholdet i innholdsplassholderne. Bruk kodesnuttgeneratoren på https://nlrsp.plugg.no/_layouts/15/ComponentHome.aspx?Url=https%3A%2F%2Fnlrsp%2Eplugg%2Eno%2F%5Fcatalogs%2Fmasterpage%2FKatalogelement%2DLeverand%C3%B8rer%2Easpx til å opprette og tilpasse flere innholdsplassholdere og nyttige SharePoint-enheter. Deretter kopierer og limer du dem inn som HTML-kodesnutter i HTML-koden. Alle oppdateringer i denne filen synkroniseres automatisk til det tilknyttede hoveddokument.
    <!DOCTYPE html[]>
    <html class="no-js" lang="en" xmlns:mso="urn:schemas-microsoft-com:office:office" xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882">
    	<meta charset="UTF-8">
    This is driving us nuts...

    Thursday, February 27, 2014 2:38 PM
  • What about the master though? Is that HTML5 as well and have the same declaration?

    The head in the page layout are only for the design mode if I'm not mistaken. You're only using what's in the content placeholders in combination with the master.

    Verify that the format (html4/5) matches the correct declaration in the master.

    Original post since I deleted it thinking it was not relevant:

    I'm a fellow Scandinavian and checked how we did with special characters, we had one page layout with a special character. In our visual studio solution we're use &aring; (å) and UTF8. So I downloaded a copy of the deployed html-file from _catalogs and saw that it had been replaced with å and changed to UTF8 (without BOM). I tried adding ø as well and checked with both UTF8 and UTF8 (without BOM) and it showed correctly in the web page.

    I forced the encoding to West european (Windows) in the web browser and the ø appears instead. So I'm guessing that you perhaps you should add <meta charset="utf-8" />, or the correct alternative if not html5, to the head. The ootb-master should be correct:

    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />

    I'm thinking your encoding/escapes are converted as well to ø and that the characters generated in CatalogItemReuseWebPart are escaped properly leaving "your" page layouts characters messed up but the output from the fields correct.

    • Edited by mellinge Thursday, February 27, 2014 2:51 PM Added original post
    Thursday, February 27, 2014 2:47 PM
  • Thanks.

    We will look into that.

    Thursday, February 27, 2014 3:01 PM
  • From our master in the VS-solution (UTF-8):

    <?xml version="1.0" encoding="utf-8"?><!--SPG:
    <html lang="sv" xmlns:mso="urn:schemas-microsoft-com:office:office" xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882">
            <meta charset="utf-8">

    From our page layout in the VS-solution (UTF-8):

    <?xml version="1.0" encoding="utf-8"?><!--SPG:
    <!DOCTYPE html[]>
    <html lang="en" xmlns:mso="urn:schemas-microsoft-com:office:office" xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882">
            <title>Blank sida hel</title>
            <!--MS:<asp:ContentPlaceHolder ID="PlaceHolderPageLayoutBottom" runat="server">-->
                <div class="span12 rightnow">
    				Just nu p&aring; {...}
                <!--SPM:<SharePoint:DelegateControl runat="server" ControlId="DelegatePageLayoutBottom" />-->

    We don't even have any meta tags in our page layouts (and the wrong language-attribute).

    Have you used Fiddler2 to check that no rewrites are done (reverse proxy or modified IIS-settings etc)?

    We receive the following output:

    Content-Type: text/html; charset=utf-8
    <html lang="sv" dir="ltr">
        <head><meta charset="utf-8" />{...}
                <div class="span12 rightnow">
    				Just nu på {...}

    As you can se the html entity (&aring;) is not present in the final response since it seems to have been converted in the html -> aspx process.

    • Edited by mellinge Thursday, February 27, 2014 3:20 PM
    Thursday, February 27, 2014 3:15 PM
  • We have checked that both the master page and the page layout are according to the structure you have pasted.

    We have tried with and without charset declaration in bot the the page layout and the master page file. We even disconnected and reconnected the catalog to get fresh automatically created page layout files (which are conforming with what you have pasted above). We even tried with the plain seattle and oslo templates.

    We still get the #¤%& corrupted characters.

    I have checked with fiddler2 and it states that the content delivered is utf-8. I have also checked with Firefox that the page actually is interpreted as utf-8. The characters are messed up in the 

    I thinking that there could be some unwanted caching in IIS so I have done a lot of ISSResets - no effect.

    I have also opened both page layout and master page aspx files in Notepad++ to check encoding - utf-8 without BOM. Nothing there either.

    I'm wondering if we have overseen some language related setting somewhere in the config or on the server but this was working fine last week until we deleted the web app and created a new one in order to get rid of a Content Type Hub related error.

    I have to get some sleep now to start with a fresh head tomorrow. We have saying: "If everything else fails, then JQuery..." - but that is a dirty, unwanted solution...

    Thursday, February 27, 2014 7:49 PM
  • Have you double checked the IIS site's .NET Globalization settings.

    File should be set to UTF-8.IIS Settings .NET Globalization

    Perhaps something went wrong in the provisioning when creating the new web application?

    If you check the SharePoint Web Services it's set to Windows-1252 which confirms your previous stack overflow reference I guess.

    Friday, February 28, 2014 8:48 AM
  • Problem solved. I'll update with the solution in a few minutes.

    Friday, February 28, 2014 8:55 AM
  • I can confirm I can reproduce the problem when setting it to windows1252. All dynamically generated content is correctly escaped but my å now becomes Ã¥. Hopefully this solves your problems!

    As a side note, I've seen problems in a farm when only one of the web server has got a IIS site created, when creating a new web application, without any error indications.

    • Edited by mellinge Friday, February 28, 2014 9:02 AM Added side note
    Friday, February 28, 2014 8:59 AM
  • Excellent, thanks!. I checked the top level globalization setting i IIS and this was set to Windows-1252.

    However, our work around solution was to add the 

    <meta charset="utf-8">

    declaration inside the first MS-statement inside the body tag of the page layout HTML file like this:

    <?xml version="1.0" encoding="utf-8"?><!--SPG:
    <!DOCTYPE html[]>
    <html class="no-js" lang="en" xmlns:mso="urn:schemas-microsoft-com:office:office" xmlns:msdt="uuid:C2F41010-65B3-11d1-A29F-00AA00C14882">
            <title>Some title
            <!--MS:<asp:ContentPlaceHolder ID="PlaceHolderTopNavBar" runat="server">-->
    		<meta charset="utf-8">
                <!--MS:<SharePoint:AspMenu ID="TopNavigationMenu" runat="server" EnableViewState="false" DataSourceID="topSiteMap" AccessKey="&#60;%$Resources:wss,navigation_accesskey%&#62;" UseSimpleRendering="true" UseSeparateCss="false" Orientation="Horizontal" StaticDisplayLevels="1" AdjustForShowStartingNode="false" MaximumDynamicDisplayLevels="3" SkipLinkText="">-->

    Strange, but it worked.

    Actually we tried to set the globalization settings in the web.config files of the different IIS sites running in the webapp yesterday like this:

          <add name="SPWindowsClaimsAuthentication" type="Microsoft.SharePoint.IdentityModel.SPWindowsClaimsAuthenticationHttpModule, Microsoft.SharePoint.IdentityModel, Version=, Culture=neutral, PublicKeyToken=71e9bce111e9429c" />
        <globalization requestEncoding="utf-8" responseEncoding="utf-8" culture="nb-NO" fileEncoding="utf-8"/>
        <!--<globalization fileEncoding="utf-8" />-->
        <compilation batch="false" debug="false">

    As far as I can understand this should have the same effect as setting this in IIS.

    It seems that the IIS setting actually overrides whatever you put in web.config for globalization options.

    We have updated the IIS setting now to UTF-8 and done a IIS Reset. At the moment we get corrupted characters still, but now the existing aspx-files probably is corrupted and needs to be replaced with fresh ones. I will do a catalog dis-/reconnect.

    I'll update with whatever we end up with as the final solution. Anyway, very grateful for the invaluable help from a fellow Scandinavian :-)

    Friday, February 28, 2014 9:23 AM
  • Sounds odd that that the fix with the meta-tag in the wrong location solved the problem. I find it more likely that you cleared the Page Output Cache or something similar when doing the change :) But anyway, I hope you get those ø rolling.
    Friday, February 28, 2014 12:17 PM
  • Quick Summary

    • we have fixed the file encoding setting in IIS - utf-8 specified at server level and on all webapps running in the IIS installation
    • Still seems to be some issues with encoding necessary in the page layout files - but this could be connected to how our masterpage looks like

    For the time beeing it seems that we have to add the charset declaration below the

    ContentPlaceHolder id="PlaceHolderAdditionalPageHead"

    But this could be due to some caching of some sort. We disconnect catalogs, delete the category and item pages both via GUI and windows explorer (/_catalogs/masterpage) and then reconnect the catalog. When the charset declaration is just below the head tag in the masterpage it doesn't work. When the charset declaration is inside the ContenPlaceHolder it is added to Page Layout files automatically generated by the catalog connection and then it works.

    But it's rolling now :-) A lot of moving parts in Sharepoint so there is a lot of factors that could cause it. Anyway, we have to focus on tasks instead of explanations now.

    Thanks again!

    Friday, February 28, 2014 12:24 PM
  • Hi there,

    we had the same issue with German characters in a page layout HTML file. Basically we tried to add a display title for a refiner web part snippet. After some test with HTML encoding we figured out, that JavaScript encoding is the right way to place special characters in display template snippets.

    Ä = \u00c4
    ä = \u00e4
    Ö = \u00d6
    ö = \u00f6
    Ü = \u00dc
    ü = \u00fc
    ß = \u00df 

    Friday, August 15, 2014 3:28 PM