30 August 2013
Last November, I was helping someone consume a WCF Web Service with PHP (in the imaginatively named Consuming a WCF Web Service from PHP). After jumping through some hoops (and reading a lot of unhelpful and/or misleading information on the web) it was working; requests that relied on type names being specified were being accepted, gzip support was being enabled, even some useful debug information was being made available for when problems were encountered. All was well. But there was something that was bugging me for a long time that I only quite recently was able to address -
Why does the PHP SoapClient so belligerently throw away the type names of response objects?
It has knowledge of the type name since it must process the response data to populate the associative arrays that represent this data. But the names of the response types are apparently then cast forever into the ether, never to exposed to me. After all, I'm using PHP - I don't want no stinkin' types!
I feel I should probably explain why I care so much. To be fair, I imagine that in a large number of cases the type name of the returned data really isn't important. If, for example, I'm querying the Twitter API for a set of Statuses then I know the form of the returned data (and since it returns JSON, there are no type names in the responses!). And for a lot of services, I imagine the form of the returned data will be identical from one result to another and, in many of the cases where the forms vary, a form of "property sniffing" will deal with it; does this result have this particular property along with all of the common ones? If so, save it or use it or do whatever with it.
But there are cases where this isn't enough. In that earlier post, the example was a web method "GetHotels" which returned hotel data for results that matched a particular set of filters (in that case, the type names were important for the request since an array of filters was specified, each filter was a particular WCF class - without the type names, the service couldn't deserialise the request).
Each of the returned hotels has data such as Categories, Awards, and Facilities but only the keys of these Categories, Awards and Facilities are returned. There is a separate web method "GetMetaData" that maps these keys onto names. A language can be specified as part of the meta data request so that the names are provided in the correct language.
Some of the meta data types may have additional data, such as an optional ImageUrl for Awards. Categories can be grouped together, so Categories such "Budget Hotel", "Boutique Hotel" and "Garden Hotel" are all considered to be part of the Category Group "Hotel" whilst "Guest House", "Farmhouse" and "Inn" are all considered part of the "Bed & Breakfast" Category Group.
The natural way to express this in a WCF Web Service (making use of wsdl-supported complex types) is something like the following -
[ServiceContract]
public class HotelService
{
[OperationContract]
public MetaDataEntry[] GetMetaData(MetaDataRequest request)
{
..
}
}
[DataContact]
public class MetaDataRequest
{
[DataMember]
public string APIKey { get; set; }
[DataMember]
public string LanguageCode { get; set; }
[DataMember]
public MetaDataType[] MetaDataTypes { get; set; }
}
public enum MetaDataType
{
Award,
Category,
CategoryGroup,
Facility
}
[KnownType(AwardMetaDataEntry)]
[KnownType(CategoryMetaDataEntry)]
[KnownType(CategoryGroupMetaDataEntry)]
[KnownType(FacilityMetaDataEntry)]
[DataContract]
public abstract class MetaDataEntry
{
[DataMember(IsRequired = true)]
public int Key { get; set; }
[DataMember]
public string Name { get; set; }
}
[DataContract]
public class AwardMetaDataEntry : MetaDataEntry
{
[DataMember]
public string ImageUrl { get; set; }
}
[DataContract]
public class CategoryMetaDataEntry : MetaDataEntry
{
[DataMember(IsRequired = true)]
public int CategoryGroup { get; set; }
}
[DataContract]
public class CategoryGroupMetaDataEntry : MetaDataEntry { }
[DataContract]
public class FacilityMetaDataEntry : MetaDataEntry { }
The MetaDataRequest allows me to specify which types of meta data that I'm interested in.
So, feasibly, if I wanted to build up a set of Categories to map the keys from the Hotels onto, I could make a request for just the meta data for the Categories. If I then want to map those Categories onto Category Groups, I could make a request for the Category Group meta data.
But why shouldn't I be able to request all of the meta data types, loop through them and stash them all away for future reference all in one go? I could do this easily enough with a .net client. Or a Java client. But, by default, PHP refuses to allow a distinction to be made between a CategoryGroupMetaDataEntry and a FacilityMetaDataEntry since they have the same structure and PHP won't tell me type names.
Well.. that's not strictly true. PHP does have some means to interrogate type names; the methods "gettype" and "get_class". If you define a class in your PHP code and pass an instance of it to the "get_class" method, you will indeed get back the name of that class. "get_class" may only be given an argument that is an object, as reported by the "gettype" method (see the get_class and gettype PHP documentation).
But if we try this with the web service call -
$client = new SoapClient(
"http://webservice.example.com/hotelservice.svc?wsdl",
array(
"compression" => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP,
"trace" => 1
)
);
$metaDataTypes = $client->GetMetaData(
array(
"request" => array(
"ApiKey" => "TestKey",
"Language" => 1,
"MetaDataTypes" => array(
"MetaDataTypeOptions" => array(
"Award",
"Category",
"CategoryGroup",
"Facility"
)
)
)
)
);
we can loop through the returned data and use get_class to find out that.. they are all apparently "StdObject".
This is what I meant by the type names being "thrown away".
In some cases we can work around this.
For example, to guess that a result is an AwardMetaDataEntry we could try
if (property_exists($metaDataValue, "ImageUrl")) {
and work on the basis that if it exposes an "ImageUrl" property that it is AwardMetaDataEntry.
But this won't work for differentiating between a CategoryGroupMetaDataEntry and a FacilityGroupMetaDataEntry since those response types have no structural differences.
It turns out that the SoapClient does offer a way to get what we want, so long as we don't mind declaring PHP classes for every response type that we're interested in.
class MetaDataEntry
{
public $Key;
public $Name;
}
class AwardMetaDataEntry extends MetaDataEntry
{
public $ImageUrl;
}
class CategoryMetaDataEntry extends MetaDataEntry
{
public $CategoryGroup;
}
class CategoryGroupMetaDataEntry extends MetaDataEntry { }
class FacilityMetaDataEntry extends MetaDataEntry { }
As we can see in the PHP SoapClient documentation, one of the options that can be specified is a "classmap" -
This option must be an array with WSDL types as keys and names of PHP classes as values
It's a way to say that particular response types should be mapped to particular PHP classes - eg.
$client = new SoapClient(
"http://webservice.example.com/hotelservice.svc?wsdl",
array(
"compression" => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP,
"trace" => 1,
"classmap" => array(
"AwardMetaDataEntry" => "AwardMetaDataEntry",
"CategoryMetaDataEntry" => "CategoryMetaDataEntry",
"CategoryGroupMetaDataEntry" => "CategoryGroupMetaDataEntry",
"FacilityMetaDataEntry" => "FacilityMetaDataEntry"
)
)
);
Now when we loop through the response values and call get_class we get the correct names. Success!
(In the above code I've named the PHP classes the same as the WSDL types but, since the mappings all have to be individually specified, the class names don't have to be the same. The properties, on the other hand, do have to match since there is no facility for custom-mapping them. Any classes that don't have a mapping will continue to be translated into objects of type StdObject).
It may well be that this is far from news for many seasoned PHP Developers but when I described the situation (before finding out about the "classmap" option) to someone I was told was experienced and competent they had no suggestion in this direction.
To be honest, I'm not sure how I came across this in the end. If you know that there exists an option to map classes with the SoapClient then it's easy to find; but with only a vague idea that I wanted it to stop throwing away type names, it took me lots of reading and clutching at straws with search terms. Interestingly, even with this knowledge, I'm still unable to find an article that describes the specific problem I've talked about here.. so maybe it really is just me that has encountered it or cares about it!
Posted at 00:03
9 November 2012
For the Web Service that I've been developing at work, I was asked to support a Developer who was trying to consume it for a project. The problem being that he was developing in PHP and had never connected to a .Net Web Service before - at least not knowingly; the APIs he'd integrated with before would all communicate in JSON. The other half of the problem is that I've never developed anything in PHP! In fact I'd not written a line in my life before last week.
From what I'm led to understand of PHP it's somewhat of a mess of 1000s of functions with inconsistent names, signatures and approaches. It's dynamically typed and may or may not have namespaces in any usable manner. But it's got an enormous set of well-established libraries available for everything under the sun. So this should be a walk in the park, right??
Below is a simplified version of what we're working with; the idea that you could search for Hotels that meet various criteria, where the criteria can be built up with the application of multiple Filters - at least one must be specified but multiple may be, in which case Hotels must meet the criteria in all Filters in order to be returned.
[ServiceContract]
public class HotelService
{
[OperationContract]
public Hotel[] GetHotels(HotelSearchRequest request)
{
..
}
}
[DataContact]
public class HotelSearchRequest
{
[DataMember]
public string APIKey { get; set; }
[DataMember]
public Filter[] Filters { get; set; }
}
[KnownType(CategoryFilter)]
[KnownType(ProximityFilter)]
[DataContract]
public abstract class Filter { }
[DataContract]
public class CategoryFilter : Filter
{
[DataMember]
public int[] CategoryKeys { get; set; }
}
[DataContract]
public class ProximityFilter : Filter
{
[DataMember]
public CoordinateDetails Centre { get; set; }
[DataMember(IsRequired = true)]
public int MaxDistanceInMetres { get; set; }
}
We have to mess about a bit with KnownType declarations in the right place in the service contract but once that's all sorted the service is easy to consume from a .Net WCF Client, all of the inheritance is easily understood (as they're documented in the xsd that the service exposes) and queries are easy to construct.
Getting things working in PHP is another matter, however. For someone who doesn't know what he's doing, at least! All of the basic examples suggest something along the lines of:
$client = new SoapClient(
"http://testhotelservice.com/HotelService.svc?wsdl",
array("trace" => 1)
);
$data = $client->GetHotels(
array(
"request" => array(
"ApiKey" => "{KeyGoesHere}",
"Filters" => array(
// TODO: What to write here?!
)
)
)
);
It seems like the SoapClient can do some sort of magic with what are presumably associative arrays to build up the web request. All looks good initially for declaring data for the "request" argument of the GetHotels method; we set the "ApiKey" property of the request to an appropriate string.. the SoapClient must be doing something clever with the wsdl to determine the xml to generate, which must include the type name of the request to specify. But if the type names are intended to be hidden, how am I going to specify them when I build the "Filters" array?? I can't just carry on with this type-name-less associated-array approach because there will be no way for the request to know that I want the CategoryFilter or the ProximityFilter (or any other Filter that might be available).
Hmmm...
More googling brings me to discover the SoapVar class for use with the SoapClient. If we do the following:
$client = new SoapClient(
"http://testhotelservice.com/HotelService.svc?wsdl",
array("trace" => 1)
);
$data = $client->GetHotels(
array(
"request" => array(
"ApiKey" => "{KeyGoesHere}",
"Filters" => array(
new SoapVar(
array(
"CategoryKeys" => array(1, 2, 3)
),
SOAP_ENC_OBJECT,
"CategoryFilter",
"http://schemas.datacontract.org/2004/07/DemoService.Messages.Requests"
)
)
)
)
);
Then we are able to include information about the type. Progress! The namespace string specified references the C# namespace of the CategoryFilter class.
As with so many things, it looks all so easy. But I didn't know what exactly I should be searching for, getting this far took me quite a while - and the resource out there explaining this are thin on the ground! With the maturity of both PHP and WCF I would have thought that information about calling into WCF Services from PHP in this manner would be much readily available!
While I was at it, I thought I'd dig a little further. When I first communicated with this other Developer, I asked him to send me a trace of the requests that were being generated by his PHP code, using Fiddler or something. This information was not forthcoming and when I was running test PHP scripts on my local machine the requests weren't being captured by Fiddler anyway. But I found these handy methods:
$client->__getLastRequest()
$client->__getLastRequestHeaders()
which will retrieve the sent content, ideal for looking into exactly what messages were being generated! These only work if the "trace" => "1" argument is specified when the SoapClient is instantiated. Which explained to me what that was for, which was nice :)
The next issue I had was another one that I thought would be beyond easy to solve, with easy-to-follow and accurate information all over the place. I was wrong again! At least, my first searches didn't bring me immediately to the answer :(
A lot of resources suggest the following:
$client = new SoapClient(
"http://testhotelservice.com/HotelService.svc?wsdl",
array(
"compression" => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP | 9,
"trace" => 1
)
);
and some suggest this variation (quoting the compression value):
$client = new SoapClient(
"http://testhotelservice.com/HotelService.svc?wsdl",
array(
"compression" => "SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP | 9",
"trace" => 1
)
);
These do not work. The first results in a "can't uncompress compressed response" after a delay which makes me think that it's doing work. The latter does cause any error but also doesn't include the "Accept-encoding: gzip" HTTP header that I'm looking for.
They both feel wrong, anyway; presumably the 9 relates to gzip compression level 9. The compression level should surely be set on the server only, not referenced by the client?? And what are these SOAP_COMPRESS_ACCEPT and SOAP_COMPRESSION_GZIP values? These values are just numeric constants, it turns out, which are OR'd together in the first variation. But what's the 9 for; is it supposed to be there at all; is it some other mystery constant?? And surely the quoted version is incorrect unless PHP has some mad string rules that I don't know about (totally possible with my knowledge of PHP but not the case here! :)
The correct version is simply:
$client = new SoapClient(
"http://testhotelservice.com/HotelService.svc?wsdl",
array(
"compression" => SOAP_COMPRESSION_ACCEPT | SOAP_COMPRESSION_GZIP,
"trace" => 1
)
);
Again, oh so simple yet so much harder to come to in the end than it should have been.
A final note, that actually was commonly documented and pointed out, was that if you are developing against a service that is still in flux that you should disable wsdl caching in PHP. This will affect performance as the wsdl will be retrieved on each request (I presume), but it may prevent some headaches if the contract changes. I changed a value in the php.ini file but apparently it can also be done in the PHP script with:
ini_set("soap.wsdl_cache_enabled", 0);
(Courtesy of a StackOverflow answer).
This may well be simple stuff to the real PHP developers out there but since I struggled, maybe this post will help others in the future with this particular problem!
Posted at 23:03
Dan is a big geek who likes making stuff with computers! He can be quite outspoken so clearly needs a blog :)
In the last few minutes he seems to have taken to referring to himself in the third person. He's quite enjoying it.