TDS 5.5: Quick Shoutout

This is a really quick and short shout out to the TDS 5.5 Delta Deploy feature. I’ve been using the beta version of TDS 5.5 for a few weeks, and it's made quite the impact already. I love it! Check out this before and after of my deployment package:

Last week, our production deployment package was 2,040KB with all items included, and installation took about 3 minutes.

This week’s deployment package was only 171KB and it went through in approximately 20 seconds as it only included the items my team worked on for the current release.

TDS 5.5 will be available at the end of March, 2016. Contact Hedgehog Development for more info! 

Basics: How to publish an item programmatically to all targets in Sitecore

Here's a tidbit that I found myself googling about and I couldn't find an answer to.. How to publish an item to all available publishing targets.

I needed to do this to avoid hard-coding database names when publishing programmatically. So I went in to see how Sitecore does it from the ribbon command (Sitecore.Shell.Framework.Commands.PublishNow)

After some digging around in the Kernel, I ended up with something along these lines:

 Database contextDatabase = Sitecore.Context.Database;  
 Item itemToPublish = contextDatabase.GetItem("/sitecore/content/home"); //some item that needs to be published  
 //get all the available targets  
 List<Database> databases = new List<Database>();  
 ItemList targets = PublishManager.GetPublishingTargets(contextDatabase);  
 foreach (Item targetItem in targets)  
 {  
  Database database = Factory.GetDatabase(targetItem[FieldIDs.PublishingTargetDatabase]);  
   if (database != null)  
   {  
     databases.Add(database);  
   }  
 }  
 List<Language> languages = new List<Language>();  
 languages.Add(itemToPublish.Language);  
 //invoking the static PublishManager.PublishItem  
 PublishManager.PublishItem(itemToPublish, databases.ToArray(), languages.ToArray(), false, true);  

If working within the Sitecore Client, you will want to use Sitecore.Context.ContentDatabase instead of Sitecore.Context.Database

Also, if you want to publish in all languages, you can use LanguageManager.GetLanguages(contextDatabase) instead.

Sitecore Web Forms for Marketers as a Service

Customizing the WFFM module for Sitecore has provided post material for quite a few blogs, including this one. It's a favorite among clients, and it's fairly extensible. However, we all know that once in a while, we get the occasional request for a marketing form, which the module simply does not work for. My last endeavor into forms included a fancy animated multiple step wizard with interdependent fields, which would be an immense undertaking to implement with WFFM versus a simple custom .net form.

So the question became, how do I combine the value we get out of WFFM setup, save actions and reporting with the amazing designs and form flowcharts, which the design teams come up with and which our front end developers code into clean and beautiful html5. Especially in cases where we've already developed quite a few save actions that post to various client CRMs, create leads, and talk to third parties. Who wants to recode all that for a single custom form?

So I started working towards a concept, which would provide an endpoint for WFFM form submissions. From anywhere. It is fairly simple and straight forward to implement (wait till we get to the code part), but it becomes powerful in that it allows developers to have full control over the rendered html and still take advantage of the WFFM save actions.

Step 1. Run WFFM save actions on submitted form data.

We'll need to define a couple of classes for consumers of the WFFM service

   public class FormData
    {
        public string FormId { get; set; }

        public IEnumerable<FormField> Fields { get; set; } 
    }

    public class FormField
    {
        public string FieldName { get; set; }

        public string FieldValue { get; set; }
    }

And the meaty part - the FormProcessor, which is responsible for running the WFFM save actions, and of course has a dependency on the Sitecore and WFFM assemblies. With the help of a reflection tool, we can imitate what WFFM does behind the scenes here:

    public class FormProcessor
    {
        public FormProcessorResult Process(FormData data)
        {
            FormProcessorResult result = new FormProcessorResult();

            if (string.IsNullOrEmpty(data.FormId))
            {
                result.Success = false;
                result.ResultMessage = "Invalid Form Id";
            }
            else
            {
                bool failed = false;

                ID formId = new ID(data.FormId);
                FormItem formItem = FormItem.GetForm(formId);

                if (formItem != null)
                {
                    //Get form fields of the WFFM
                    FieldItem[] formFields = formItem.FieldItems;

                    //Create collection of fields
                    List<AdaptedControlResult> adaptedFields = new List<AdaptedControlResult>();
                    foreach (FormField field in data.Fields)
                    {
                        FieldItem formFieldItem = formFields.FirstOrDefault(x => x.Name == field.FieldName);
                        if (formFieldItem != null)
                        {
                            adaptedFields.Add(GetControlResult(field.FieldValue, formFieldItem));
                        }
                        else
                        {
                            // log and bail out
                            Log.Warn(string.Format("Field Item {0} not found for form with ID {1}", field.FieldName, data.FormId), this);
                            failed = true;
                            result.Success = false;
                            result.ResultMessage = string.Format("Invalid field name: {0}", field.FieldName);
                            break;
                        }
                    }

                    if (!failed)
                    {
                        // Get form action definitions
                        List<ActionDefinition> actionDefinitions = new List<ActionDefinition>();
                        ListDefinition definition = formItem.ActionsDefinition;
                        if (definition.Groups.Count > 0 && definition.Groups[0].ListItems.Count > 0)
                        {
                            foreach (GroupDefinition group in definition.Groups)
                            {
                                foreach (ListItemDefinition item in group.ListItems)
                                {
                                    actionDefinitions.Add(new ActionDefinition(item.ItemID, item.Parameters)
                                                              {
                                                                  UniqueKey = item.Unicid
                                                              });
                                }
                            }
                        }

                        //Execute form actions
                        foreach (ActionDefinition actionDefinition in actionDefinitions)
                        {
                            try
                            {
                                ActionItem action = ActionItem.GetAction(actionDefinition.ActionID);
                                if (action != null)
                                {
                                    if (action.ActionType == ActionType.Save)
                                    {
                                        object saveAction = ReflectionUtil.CreateObject(action.Assembly, action.Class,
                                                                                        new object[0]);
                                        ReflectionUtils.SetXmlProperties(saveAction, actionDefinition.Paramaters, true);
                                        ReflectionUtils.SetXmlProperties(saveAction, action.GlobalParameters, true);
                                        if (saveAction is ISaveAction)
                                        {
                                            ((ISaveAction) saveAction).Execute(formId, adaptedFields, null);
                                        }
                                    }
                                }
                            }
                            catch (Exception ex)
                            {
                                // log and bail out
                                Log.Warn(ex.Message, ex, this);
                                result.Success = false;
                                result.ResultMessage = actionDefinition.GetFailureMessage();
                                failed = true;

                                break;
                            }

                        }

                        if (!failed)
                        {
                            // set successful result
                            result.Success = true;
                            result.ResultMessage = formItem.SuccessMessage;
                        }
                    }
                }
                else
                {
                    result.Success = false;
                    result.ResultMessage = "Form not found: invalid form Id";
                }
            }

            return result;
        }

        private AdaptedControlResult GetControlResult(string fieldValue, FieldItem fieldItem)
        {
            //Populate fields with values
            ControlResult controlResult = new ControlResult(fieldItem.Name, HttpUtility.UrlDecode(fieldValue), string.Empty)
            {
                FieldID = fieldItem.ID.ToString(),
                FieldName = fieldItem.Name,
                Value = HttpUtility.UrlDecode(fieldValue),
                Parameters = string.Empty
            };
            return new AdaptedControlResult(controlResult, true);
        }
    }

Step 2. Create a WebApi endpoint for clients to post to.
Now that we have the basic setup, we can create a simple API controller:

    public class WffmController : ApiController
    {
        [HttpPost]
        public IHttpActionResult Post(FormData data)
        {
            FormProcessor processor = new FormProcessor();
            FormProcessorResult result = processor.Process(data);

            if (!result.Success)
            {
                return new BadRequestErrorMessageResult(result.ResultMessage, this);
            }

            return new OkResult(this);
        }

Step 3. Register routes

var config = GlobalConfiguration.Configuration;
            config.Routes.MapHttpRoute("DefaultApiRoute",
                                     "api/{controller}/{id}",
                                     new { id = RouteParameter.Optional });

Step 4. Use with any form anywhere.

            var formFields = new List<FormField>();
            formFields.Add(new FormField
            {
                FieldName = "First Name",
                FieldValue = data.FirstName
            });
            formFields.Add(new FormField
            {
                FieldName = "Last Name",
                FieldValue = data.LastName
            });
            formFields.Add(new FormField
            {
                FieldName = "Email",
                FieldValue = data.Email
            });

            FormData formData = new FormData();
            formData.FormId = MY_WFFM_FORM_ITEM_ID; // form item id from Sitecore
            formData.Fields = formFields;



            using (WebClient client = new WebClient())
            {
                client.UploadString(RemoteUrl, "POST", JsonConvert.SerializeObject(formData));
            }

Cons:
- certain WFFM out-of-the-box features will be lost - validation! validation! validation!
- once created, the form needs to remain fairly immutable since the form item ID and the field items are the contract with any client that will submit data.

I hope that someone would find this useful the next time they're faced with a similar problem. 

Sitecore error with Lucene Thai Analyzer

ManagedPoolThread #1 2015:03:12 08:32:28 ERROR Exception

Exception: System.Reflection.TargetInvocationException

Message: Exception has been thrown by the target of an invocation.

Source: mscorlib

   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)

   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)

   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)

   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)

   at (Object , Object[] )

   at Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args)

   at Sitecore.Jobs.Job.ThreadEntry(Object state)

Nested Exception

Exception: System.NotSupportedException

Message: PORT ISSUES

Source: Lucene.Net.Contrib.Analyzers

   at Lucene.Net.Analysis.Th.ThaiAnalyzer.ReusableTokenStream(String fieldName, TextReader reader)

   at Lucene.Net.Index.DocInverterPerField.ProcessFields(IFieldable[] fields, Int32 count)

   at Lucene.Net.Index.DocFieldProcessorPerThread.ProcessDocument()

   at Lucene.Net.Index.DocumentsWriter.UpdateDocument(Document doc, Analyzer analyzer, Term delTerm)

   at Lucene.Net.Index.IndexWriter.UpdateDocument(Term term, Document doc, Analyzer analyzer)

   at Sitecore.ContentSearch.LuceneProvider.LuceneUpdateContext.UpdateDocument(Object itemToUpdate, Object criteriaForUpdate, IExecutionContext[] executionContexts)

   at Sitecore.ContentSearch.SitecoreItemCrawler.DoUpdate(IProviderUpdateContext context, SitecoreIndexableItem indexable)

   at Sitecore.ContentSearch.LuceneProvider.LuceneIndex.PerformUpdate(IEnumerable`1 indexableUniqueIds, IndexingOptions indexingOptions)

In a single day, we saw this error appear over 9000 times on a production environment.

From what I understand (since 7.0+) Sitecore by default provides full mapping of all available Lucene.net analyzers. They are configured under:

indexConfigurations > defaultLuceneIndexConfiguration > analyzer > param desc="map"

Based on the context of the content that's indexed/searched, Sitecore will (with reflection) figure out which mapping to use. Here’s a great post explaining execution contexts - http://www.sitecore.net/learn/blogs/technical-blogs/sitecore-7-development-team/posts/2013/08/execution-contexts-explained.aspx

So the Thai Analyzer seems to be a bit broken (read not implemented) from what I see in the Lucene.Net source. The Analyzer calls the constructor for ThaiWordFilter with a token stream and that constructor just throws the exception we see. You can decompile the Lucene.Net.Contrib.Analyzers.dll or look at the source at http://lucenenet.apache.org/.

public ThaiWordFilter(TokenStream input): base(input)
{
  throw new NotSupportedException("PORT ISSUES");
  //breaker = BreakIterator.getWordInstance(new Locale("th"));
  //termAtt = AddAttribute<TermAttribute>();
  //offsetAtt = AddAttribute<OffsetAttribute>();
}

Removing or commenting out the Thai analyzer (the below mapEntry) from the execution context mappings in the Sitecore.ContentSearch.Lucene.DefaultIndexConfiguration.config should result in indexing/searching in th-TH to fall back to the standard analyzer and will get rid of the error in your log files.

             <mapEntry type="Sitecore.ContentSearch.LuceneProvider.Analyzers.PerExecutionContextAnalyzerMapEntry, Sitecore.ContentSearch.LuceneProvider">

                <param hint="executionContext" type="Sitecore.ContentSearch.CultureExecutionContext, Sitecore.ContentSearch">

                  <param hint="cultureInfo" type="System.Globalization.CultureInfo, mscorlib">

                    <param hint="name">th-TH</param>

                  </param>

                </param>

                <param desc="analyzer" type="Sitecore.ContentSearch.LuceneProvider.Analyzers.DefaultPerFieldAnalyzer, Sitecore.ContentSearch.LuceneProvider">

                  <param desc="defaultAnalyzer" type="Lucene.Net.Analysis.Th.ThaiAnalyzer, Lucene.Net.Contrib.Analyzers">

                    <param hint="version">Lucene_30</param>

                  </param>

                </param>

              </mapEntry>

If anyone has come across this before, I'd love to hear from you!


Update: Pavel Veller (@pveller) pointed out to me that this issue has been fixed with Sitecore 7.2 Update 3. As per the release notes:

• Thai Analyzer from Lucene.Net was not fully implemented and could sometimes throw Not Supported exceptions. The analyzer has been removed from the default Lucene index configuration. The default analyzer will be used instead. (420234)

Searchable Language Selector

If you have ever worked in a Sitecore instance with a lot of languages, you may have noticed that sometimes it could be quite time consuming (and frustrating) to look for the language you need in the language picker. This isn't as much a developer problem as it is an issue for the content editors who often make edits in multiple languages. So, here's a quick and easy client-side solution.

The language selector is generated by an xml control located here: \sitecore\shell\Applications\Content Manager\Galleries\Languages\Gallery Languages.xml

A couple of modifications to add a search box, and a couple of javascript functions later, and we now have a searchable language selector:

You can find the modified control up on GitHub. Let me know what you guys think!

Update: This modification is now also available for download from the Sitecore Marketplace.

Error when rendering WFFM form

I came across an interesting WFFM exception on a production CM environment today. It turned out to be a configuration error, so I decided to share

[InvalidOperationException: folder]
   Sitecore.Form.Core.Configuration.ThemesManager.GetThemeName(Item form, ID fieldID) +434
   Sitecore.Form.Core.Configuration.ThemesManager.GetThemeUrl(Item form, Boolean deviceDependant) +270
   Sitecore.Form.Core.Configuration.ThemesManager.ScriptsTags(Item form, Item contextItem) +49
   Sitecore.Form.Core.Configuration.ThemesManager.RegisterCssScript(Page page, Item form, Item contextItem) +184
   Sitecore.Form.Web.UI.Controls.SitecoreSimpleFormAscx.OnInit(EventArgs e) +233
   System.Web.UI.Control.InitRecursive(Control namingContainer) +186
   System.Web.UI.Control.AddedControl(Control control, Int32 index) +189
   Sitecore.Form.Core.Renderings.FormRender.OnInit(EventArgs e) +846
   System.Web.UI.Control.InitRecursive(Control namingContainer) +186
   System.Web.UI.Control.InitRecursive(Control namingContainer) +291
   System.Web.UI.Control.InitRecursive(Control namingContainer) +291
   System.Web.UI.Control.InitRecursive(Control namingContainer) +291
   System.Web.UI.Control.InitRecursive(Control namingContainer) +291
   System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +2098

The method - Sitecore.Form.Core.Configuration.ThemesManager.GetThemeName(Item form, ID fieldID) - looks at the Form ID that's configured as the Forms root ID in the site definition.

string formsRootForSite = SiteUtils.GetFormsRootForSite(Context.Site);
Item item = form;
if (form.TemplateID != IDs.FormFolderTemplateID)
{
    item = form.Database.GetItem(formsRootForSite);
}
Assert.IsNotNull(item, "folder");

In my case, the configured ID did not match the actual forms folder item ID in Sitecore.

Vulnerability with using the Sitecore context search index

Mornings when production issues happen mean two things: no tea yet, and definitely no snacks until order is restored. So I was in a bit of a hurry today to find out what was going on when our default Sitecore content indexes seemed to have gone down. This was the exception thrown:

System.NullReferenceException: Object reference not set to an instance of an object. at Sitecore.ContentSearch.SitecoreItemCrawler.IsAncestorOf(Item item) at Sitecore.ContentSearch.SitecoreItemCrawler.IsExcludedFromIndex(SitecoreIndexableItem indexable, Boolean checkLocation) at Sitecore.ContentSearch.Pipelines.GetContextIndex.FetchIndex...

Here's a snippet that would cause the above error to be thrown:

Item contextItem = Sitecore.Context.Database.GetItem(SOME_ID);

if (contextItem != null)
{
    ISearchIndex index = ContentSearchManager.GetIndex(new SitecoreIndexableItem(contextItem));
}

That's just an example, but I would also see the error thrown in the __semantics field in my 'web' database.

So we're failing to retrieve the ISearchIndex... but why?

If you take a look at IsAncestorOf(Item item) (Sitecore 7.2+) using your favorite reflection tool, this is what you'll find:

// Sitecore.ContentSearch.SitecoreItemCrawler
protected virtual bool IsAncestorOf(Item item)
{
 bool result;
 using (new SecurityDisabler())
 {
  using (new CachesDisabler())
  {
   result = this.RootItem.Axes.IsAncestorOf(item);
  }
 }
 return result;
}

No null checks here, that can't be good.
The RootItem property -

// Sitecore.ContentSearch.SitecoreItemCrawler
public Item RootItem
{
    get
    {
        if (this.rootItem == null)
        {
            Database database = ContentSearchManager.Locator.GetInstance<IFactory>().GetDatabase(this.database);

            Assert.IsNotNull(database, "Database " + this.database + " does not exist");

            using (new SecurityDisabler())
            {
                this.rootItem = database.GetItem(this.Root);
            }
        }
        return this.rootItem;
    }
}

No null check here either. Fishy.

I had to dig a bit deeper to figure out where it all originates and what exactly is null.

I overwrote the Sitecore.ContentSearch.Pipelines.GetContextIndex.FetchIndex, Sitecore.ContentSearch pipeline processor to debug the code. It blew up trying to evaluate this:

   System.Collections.Generic.IEnumerable<ISearchIndex> enumerable =
                from searchIndex in ContentSearchManager.Indexes
                from providerCrawler in searchIndex.Crawlers
                where !providerCrawler.IsExcludedFromIndex(indexable)
                select searchIndex;

So the GetContextIndex pipeline tries to fetch all of the available crawlers, whose roots are ancestors of our indexable item (IsExcludedFromIndex). The ancestor check however fails on a null root item in any of the indexes.

So if you remember the good old "Root Item Not defined" error message,  it turns out this issue had the exact same cause - the RootItem for one of the search indexes was not defined in the 'web' database (i.e. the item was not yet published). Publishing is of course the quick fix.

While this is in the end a configuration issue, I still feel like the unpublished site should not be affecting ALL OF SEARCH (search based on GetContextIndex that is), which is why I'm going to leave my FetchIndex override in place and add some null checks for at least a more meaningful error message.